vsf.core module

class vsf.core.BaseVSF[source]

Bases: object

Base class for VSF model.

The volumetric stiffness field defines the stiffness of a set of Hookean springs in a 3D volume. For each spring with deformation u, VSF will generate contact force f = -k*u, where k is the stiffness of the spring.

getBBox() → Tensor[source]: Return the bounding box of the model.

getStiffness(points: Tensor) → Tensor[source]

Get stiffness at 3d points.

Parameters:: points – a tensor of shape (N,3)
Returns:: a tensor of shape (N,)
Return type:: stiffness

load(path: str)[source]: Load the VSF model parameters from a folder/file.

save(path: str)[source]: Save the VSF model parameters in a folder/file.

to(device) → BaseVSF[source]

Converts the VSF to a given device or dtype.

Note: modifies the model in-place.

class vsf.core.NeuralVSF(vsfConfig: NeuralVSFConfig, sdf: Tensor | None = None)[source]

Bases: BaseVSF

A Neural VSF model that conforms to the BaseSDF base class.

Optionally takes an SDF tensor to use as a ground truth SDF for training.

Parameters:

vsfConfig (dict) – Configuration for the Neural VSF model.
sdf (torch.Tensor, optional) – Optional SDF tensor to use as a geometry mask. The SDF values are sampled at the center of the grid cells. Cell (0,0,0) is at BBox[0] and cell (N-1, N-1, N-1) is at BBox[1].

property device

property dtype

getBBox() → Tensor[source]: Return the bounding box of the model.

getStiffness(position: Tensor) → Tensor[source]

Get stiffness at 3d points.

Parameters:: points – a tensor of shape (N,3)
Returns:: a tensor of shape (N,)
Return type:: stiffness

load(path)[source]: Load the VSF model parameters from a folder/file.

save(path)[source]: Save the VSF model parameters in a folder/file.

to(device) → NeuralVSF[source]: Converts the VSF to a given device or dtype

Bases: BaseVSF

A VSF model that consists of a set of particles.

The stiffness field stiffness is defined at each particle. The axis_mode parameter defines the symmetry of the VSF material: - If axis_mode = ‘isotropic’, then the stiffness field is a scalar. - If axis_mode = ‘axis_aligned’, then the stiffness field is a 3-vector. - If axis_mode = ‘anisotropic’, then the stiffness field is a 3x3 matrix.

Features can also be defined at each particle, which are stored in the features dictionary. Typical features include: - ‘volume’: the volume of space represented by the particle - ‘K_std’: the standard deviation of the particle’s stiffness - ‘N_obs’: the number of observations of the particle - ‘mass’: the mass of the particle - ‘damping_ratio’: the damping ratio of the particle - ‘friction’: the friction coefficient of the particle

rest_points

Nx3 matrix giving rest positions of particles

Type:: np.ndarray or torch.Tensor

axis_mode

axis mode of the stiffness field

Type:: str

stiffness

Nx1, Nx3, or Nx3x3 tensor giving the stiffness field at each particle

Type:: torch.Tensor

features

optional features of the particles, each of leading dimension N.

Type:: dict[str,torch.Tensor]

compute_forces(point_indices: LongTensor, displaced_points: Tensor) → Tensor[source]

Computes the forces on a set of points given their indices and displaced positions.

All quantities are in the local frame of the VSF.

Parameters:

point_indices (torch.LongTensor) – a length M tensor of point indices
displaced_points (torch.Tensor) – a Mx3 tensor of displaced points

Returns:

a Mx3 tensor of forces that would be applied to the VSF.

Return type:

torch.Tensor

property device: device: Returns the device of the VSF.

property dtype: dtype: Returns the dtype of the VSF.

feature_tensor(feature_names: List[str] | None = None) → Tensor[source]

Returns a tensor of features for all particles.

If feature_names is None, returns all features. Otherwise, returns only the specified features.

To avoid duplicate rest_points, when input feature_names contains ‘rest_points’, the function will return a tensor of rest_points instead of the feature tensor.

Result has shape N x F where F is the sum of selected feature sizes.

getBBox() → Tensor[source]: Return the bounding box of the model.

getStiffness(points: Tensor) → Tensor[source]

Get stiffness at 3d points.

Parameters:: points – a tensor of shape (N,3)
Returns:: a tensor of shape (N,)
Return type:: stiffness

load(path: str)[source]

Loads from one of the standard file formats:

If path is a directory, then the following files are expected:
- points.npy: Nx3 array of rest points
- K.npy: Nx1, Nx3, or Nx3x3 array of stiffnesses
- other .npy / .npz files: optional features
If path is a file, then it is a numpy file with the following fields:
- points: Nx3 array of rest points
- axis_mode: string
- K: Nx1, Nx3, or Nx3x3 array of stiffnesses
- features: dictionary of features

property num_points: int

save(path: str)[source]: Saves to either a folder or a Numpy archive (npz) file.

set_feature(name: str, feature: ndarray | Tensor)[source]: Sets / adds a feature to the VSF.

set_uniform_feature(name: str, value: float | ndarray | Tensor)[source]: Sets a uniform feature to all the VSF particles.

to(device, dtype=None) → PointVSF[source]: Converts the VSF to a given device or dtype

class vsf.core.VSFFactory(config: VSFFactoryConfig)[source]

Bases: object

A VSF factory that produces PointVSF objects according to some configuration. You can downsample, create features, etc.

Automatic usage:

config = VSFFactoryConfig(...)     #set up the desired voxel resolution, features, etc.
factory = VSFFactory(config)
for i,input in enumerate(inputs):
    #input can be a bounding box, point cloud, or other geometry
    vsf = factory.process(input)   #does everything
    #save the result
    path = f'vsf_template{i}/'
    os.path.mkdir(path)
    vsf.save(path)

downsample()[source]

Downsamples the rest points according to the given parameters.

WARNING: downsampling must be performed before features are assigned.

get() → PointVSF[source]: Retrieves the currently processed VSF

load_all()[source]

load_features(features=None, obj_feats_fns=None)[source]: Loads features from the standard files in out_dir

load_rest_pts(rest_pts_fn=None)[source]: Loads rest points from the standard file in out_dir.

Performs the entire VSF construction pipeline from a given input.

Will do the best it can to extract appropriate features.

Parameters:

input – a filename, bounding box, set of rest points, PointCloud, or klampt Geometry3D. For a bounding box, the particles will be sampled with resolution config.voxel_size. For a Geometry3D, point clouds will be converted directly. Other geometry types will be converted to a SDF at resolution config.voxel_size.
features – a dictionary of provided features.

Returns:

the instantiated VSF.

Return type:

PointVSF

save_all()[source]

save_features(features=None, obj_feats_fns=None)[source]: Saves features to the standard files in out_dir.

save_rest_pts(rest_pts_fn=None)[source]: Saves rest points to the standard file in out_dir.

set_features(name: str, feature_array: ndarray | None = None)[source]: Sets the features under name to a given array, or use the default feature constructor for that feature.

class vsf.core.VSFFactoryConfig(features: ~typing.List[str] = <factory>, fps_num: int | None = None, voxel_size: float | None = 0.005, debug: bool = False, verbose: int = 0)[source]

Bases: object

Configuration for the VSF factory process.

debug: bool = False: whether to visually debug the factory construction process

features: List[str]: the factory will produce features according to this order

fps_num: int | None = None: if present, performs Farthest Point Sampling to downsample to this number of points

verbose: int = 0: verbosity level

voxel_size: float | None = 0.005: performs voxel subsampling to this resolution

class vsf.core.VSFRGBDCameraFactory(config: VSFRGBDCameraFactoryConfig)[source]

Bases: VSFFactory

A VSF factory that produces VSF objects from RGBD camera data.

Inherits from VSFFactory.

Operates by: 1. Subsetting an RGBD point cloud by a bounding box selection 1. Creating surface points from the subsetted point cloud 2. Downsampling, if requested 3. Creates a interior points behind the point cloud to fill the volume 4. Extrapolates point cloud features into the volume

The input can be a background point cloud or even just a folder path.

bg_pcd

the whole point cloud

Type:: o3d.geometry.PointCloud

vis_pcd

the point cloud corresponding to the object, selected by the bounding box

Type:: o3d.geometry.PointCloud

bbox

the selection bounding box

Type:: o3d.geometry.AxisAlignedBoundingBox

vg_shape

the size of the voxel points used for creating interior points

Type:: tuple

get_sf_pts() → ndarray[source]: Returns points that belong to the visible surface

get_vg_pts() → ndarray[source]: Returns points that belong to occupied voxel grid cells

load_obj_all(bg_pcd_fn=None, vis_pcd_fn=None)[source]: Loads from previously saved files

load_vis_pcd(vis_pcd_fn=None)[source]

process(pcd: str | PointCloud, features: List[str] | Dict[str, ndarray] | None = None) → PointVSF[source]

Performs the entire VSF construction pipeline from a given input.

Parameters:

pcd – a filename, folder, or PointCloud object. If a folder, will load from folder/bg_pcd.pcd. If a filename, will load from that file.
features – a dictionary of provided features or a list of filenames in the given folder. If None, will use the default features.

Returns:

the instantiated VSF.

Return type:

PointVSF

save_obj_all(bg_pcd_fn=None, vis_pcd_fn=None)[source]

save_vis_pcd(vis_pcd_fn=None)[source]

set_vis_features(feat_type: str, values: ndarray | None = None)[source]

setup_obj_pts(bg_pcd_fn=None, cam_pt=None)[source]

class vsf.core.VSFRGBDCameraFactoryConfig(features: ~typing.List[str] = <factory>, fps_num: int | None = None, voxel_size: float | None = 0.005, debug: bool = False, verbose: int = 0, view: ~vsf.core.vsf_factory.ViewConfig = <factory>, bbox: ~typing.List[~typing.List[float]] | None = None, downsample_visual: bool = False, clean_visual: ~vsf.core.vsf_factory.CleanPCDConfig | None = None, depth_extrapolation: float = 1.0, depth_sig: float = 0.005)[source]

Bases: VSFFactoryConfig

Configuration for a VSF creation process that instantiates a point-based VSF from an RGBD camera’s point cloud.

bbox: List[List[float]] | None = None

clean_visual: CleanPCDConfig | None = None

depth_extrapolation: float = 1.0

depth_sig: float = 0.005

downsample_visual: bool = False

view: ViewConfig