Class PredictPoseIdentities

Namespace

Bonsai.Sleap

Assembly

Bonsai.Sleap.dll

Represents an operator that performs markerless multi-pose and identity estimation for each image in the sequence using a SLEAP model.

PredictPoseIdentities evaluates the full SLEAP model network. In addition to extracting pose information for each detected instance in the image, it also returns the inferred identity of the object, i.e. it performs inference on the top-down-id-model network.

In addition to the properties of the Pose object, the extended PoseIdentity class adds Identity property that indicates the highest confidence identity. This will match one of the class labels found in training_config.json. The IdentityScores property indicates the confidence values for all class labels.

Since we are very often only interested in the instance with the highest identification confidence we have added the operator GetMaximumConfidencePoseIdentity which returns the PoseIdentity with the highest confidence from the input PoseIdentityCollection. Moreover, by specifying a value in the optional Identity property, the operator will return the instance will the highest confidence for that particular class.

public class PredictPoseIdentities : Transform<IplImage, PoseIdentityCollection>

Inheritance

object

Combinator<IplImage, PoseIdentityCollection>

Transform<IplImage, PoseIdentityCollection>

PredictPoseIdentities

Inherited Members

Combinator<IplImage, PoseIdentityCollection>.Process(IObservable<IplImage>)

object.ToString()

object.Equals(object)

object.Equals(object, object)

object.ReferenceEquals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

Properties

CentroidMinConfidence

Gets or sets a value specifying the confidence threshold used to discard centroid predictions. If no value is specified, all estimated centroid positions are returned.

[Range(0, 1)]
public float? CentroidMinConfidence { get; set; }

Property Value

float?

ColorConversion

Gets or sets a value specifying the optional color conversion used to prepare RGB video frames for inference. If no value is specified, no color conversion is performed.

public ColorConversion? ColorConversion { get; set; }

Property Value

ColorConversion?

IdentityMinConfidence

Gets or sets a value specifying the confidence threshold used to assign an identity class. If no value is specified, the identity with highest confidence will be assigned to each pose.

[Range(0, 1)]
public float? IdentityMinConfidence { get; set; }

Property Value

float?

ModelFileName

Gets or sets a value specifying the path to the exported Protocol Buffer file containing the pretrained SLEAP model.

[FileNameFilter("Protocol Buffer Files(*.pb)|*.pb")]
public string ModelFileName { get; set; }

Property Value

string

PartMinConfidence

Gets or sets a value specifying the confidence threshold used to discard predicted body part positions. If no value is specified, all estimated positions are returned.

[Range(0, 1)]
public float? PartMinConfidence { get; set; }

Property Value

float?

ScaleFactor

Gets or sets a value specifying the scale factor used to resize video frames for inference. If no value is specified, no resizing is performed.

public float? ScaleFactor { get; set; }

Property Value

float?

TrainingConfig

Gets or sets a value specifying the path to the configuration JSON file containing training metadata.

[FileNameFilter("Config Files(*.json)|*.json|All Files|*.*")]
public string TrainingConfig { get; set; }

Property Value

string

Methods

Process(IObservable<IplImage>)

Performs markerless multi-pose and identity estimation for each image in an observable sequence using a SLEAP model.

public override IObservable<PoseIdentityCollection> Process(IObservable<IplImage> source)

Parameters

source IObservable<IplImage>

The sequence of images from which to extract the pose identities.

Returns

IObservable<PoseIdentityCollection>

A sequence of PoseIdentityCollection objects representing the pose identities extracted from each image in the source sequence.

See Also

PredictCentroids

PredictPoses

PredictSinglePose

GetBodyPart

GetMaximumConfidencePoseIdentity