evalio

Tool for evaluating Lidar-Inertial Odometry pipelines on open-source datasets

Usage:

$ evalio [OPTIONS] COMMAND [ARGS]...

Options:

• -D, --data-dir <path>: Directory to store downloaded datasets.
• -M, --module <str>: Custom module to load (for custom datasets or pipelines). Can be used multiple times.
• -V, --version: Show version and exit.
• --install-completion: Install completion for the current shell.
• --show-completion: Show completion for the current shell, to copy it or customize the installation.
• --help: Show this message and exit.

Commands:

• dl: Download datasets
• rm: Remove dataset(s)
• filter: Filter rosbag dataset(s) to only include...
• ls: List dataset and pipeline information
• run: Run pipelines on datasets
• stats: Evaluate the results of experiments.

evalio dl

Download datasets

Usage:

$ evalio dl [OPTIONS] {datasets}...

Arguments:

• datasets...: The dataset(s) to use [required]

Options:

• --help: Show this message and exit.

evalio rm

Remove dataset(s)

If --force is not used, will ask for confirmation.

Usage:

$ evalio rm [OPTIONS] {datasets}...

Arguments:

• datasets...: The dataset(s) to use [required]

Options:

• -f, --force: Force deletion without confirmation
• --help: Show this message and exit.

evalio filter

Filter rosbag dataset(s) to only include lidar and imu data. Useful for shrinking disk size.

Usage:

$ evalio filter [OPTIONS] {datasets}...

Arguments:

• datasets...: The dataset(s) to use [required]

Options:

• -f, --force: Force deletion without confirmation
• --help: Show this message and exit.

evalio ls

List dataset and pipeline information

Usage:

$ evalio ls [OPTIONS] {kind}:<datasets|pipelines>

Arguments:

• kind:<datasets|pipelines>: The kind of object to list [required]

Options:

• -s, --search <str>: Fuzzy search for a pipeline or dataset by name
• -q, --quiet: Output less verbose information
• --show-hyperlinks: Output full links. For terminals that don't support hyperlinks (OSC 8).
• --help: Show this message and exit.

evalio run

Run pipelines on datasets

Usage:

$ evalio run [OPTIONS]

Options:

• -c, --config <path>: Config file to load from
• -d, --datasets <str>: The dataset(s) to use
• -p, --pipelines <str>: The pipeline(s) to use
• -o, --output <path>: Output directory to save results
• -l, --length <int>: Number of scans to process for each dataset
• -v, --visualize: Visualize the results via rerun
• -s, --show <parse_vis>: Show visualization options (m: map, i: image, s: scan, f: features). Automatically implies -v.
• --rerun-failed: Rerun failed experiments. If not set, will skip previously failed experiments.
• --help: Show this message and exit.

evalio stats

Evaluate the results of experiments.

Usage:

$ evalio stats [OPTIONS] {directories}...

Arguments:

• directories...: Directory of results to evaluate. [required]

Options:

• -v, --visualize: Visualize results.
• -S, --sort <str>: Sort results by the name of a column. Defaults to RTEt.
• -r, --reverse: Reverse the sorting order. Defaults to False.
• -f, --filter <str>: Python expressions to filter results rows. 'True' rows will be kept. Example: --filter 'RTEt < 0.5'
• --only-complete: Only show results for trajectories that completed.
• --only-failed: Only show results for trajectories that failed.
• -h, --hide <str>: Columns to hide, may be repeated.
• -s, --show <str>: Columns to force show, may be repeated.
• --print-columns: Print the names of all available columns.
• --w-meters <float>: Window size in meters for RTE computation. May be repeated. Defaults to 30m.
• --w-seconds <float>: Window size in seconds for RTE computation. May be repeated. Defaults to none.
• --metric <mean|median|sse>: Metric to use for ATE/RTE computation. Defaults to sse. [default: sse]
• -l, --length <int>: Specify subset of trajectory to evaluate.
• --help: Show this message and exit.