Class PredictSinglePose
Represents an operator that performs markerless, single instance, pose estimation for each image in the sequence using a SLEAP model.
PredictSinglePose runs the single_instance model. Most Bonsai.SLEAP operators support the detection of multiple instances for each incoming frame. However, there are advantages in both performance and flexibility of identifying a single object in the frame when alternative pre-processing methods for identifying cropped regions are available.
Note
Since the centroid detection step is not performed by the network, the operator expects an already centered instance on which it will run the pose estimation. This operator will always return a single output per incoming frame, even if no valid instances are detected.
The following example workflow highlights how combining basic computer-vision algorithm for image segmentation for centroid detection with SLEAP-NN pose estimation may result in >2-fold increases in performance relative to PredictPoses operator. In this example, the first part of the workflow segments and detects the centroid positions (output of BinaryRegionAnalysis) of all available objects in the incoming frame, which are then combined with the original image to generate centered crops (CropCenter). These images are then pushed through the network that will perform the pose estimation step of the process.
Finally, it is worth noting that PredictSinglePose offers two input overloads. When a sequence of single images is provided, the operator will output a corresponding sequence of Pose objects. Since the operator skips the centroid-detection stage, it won't embed a Centroid field in the returned Pose.
Alternatively, a batch mode can be accessed by providing a sequence of batches (arrays) of images to the operator. In this case, the operator returns a sequence of PoseCollection objects, one for each input frame. This latter overload can result in dramatic gains in throughput relative to processing single images.
public class PredictSinglePose : Transform<IplImage, Pose>- Inheritance
-
PredictSinglePose
- Inherited Members
Properties
- ColorConversion
Gets or sets a value specifying the optional color conversion used to prepare RGB video frames for inference. If no value is specified, no color conversion is performed.
- ExecutionProvider
Gets or sets the ONNX runtime execution provider used to perform model inference.
- InputSize
Gets or sets a value specifying a target size used to resize video frames for inference. If no value is specified, no resizing is performed.
- ModelFileName
Gets or sets a value specifying the path to the exported ONNX file containing the pretrained SLEAP model.
- PartMinConfidence
Gets or sets a value specifying the confidence threshold used to discard predicted body part positions. If no value is specified, all estimated positions are returned.
Methods
- Process(IObservable<IplImage[]>)
Performs markerless, single instance, batched pose estimation for each array of images in an observable sequence using a SLEAP model.
- Process(IObservable<IplImage>)
Performs markerless, single instance, pose estimation for each image in an observable sequence using a SLEAP model.