evalio

Tool for evaluating Lidar-Inertial Odometry pipelines on open-source datasets

Usage:

$ evalio [OPTIONS] COMMAND [ARGS]...

Options:

• -D, --data-dir PATH: Directory to store downloaded datasets.
• -M, --module TEXT: Custom module to load (for custom datasets or pipelines). Can be used multiple times.
• -V, --version: Show version and exit.
• --install-completion: Install completion for the current shell.
• --show-completion: Show completion for the current shell, to copy it or customize the installation.
• --help: Show this message and exit.

Commands:

• dl: Download datasets
• rm: Remove dataset(s)
• filter: Filter rosbag dataset(s) to only include...
• ls: List dataset and pipeline information
• run: Run pipelines on datasets
• stats: Evaluate the results of experiments.

evalio dl

Download datasets

Usage:

$ evalio dl [OPTIONS] DATASETS...

Arguments:

• DATASETS...: The dataset(s) to use [required]

Options:

• --help: Show this message and exit.

evalio rm

Remove dataset(s)

If --force is not used, will ask for confirmation.

Usage:

$ evalio rm [OPTIONS] DATASETS...

Arguments:

• DATASETS...: The dataset(s) to use [required]

Options:

• -f, --force: Force deletion without confirmation
• --help: Show this message and exit.

evalio filter

Filter rosbag dataset(s) to only include lidar and imu data. Useful for shrinking disk size.

Usage:

$ evalio filter [OPTIONS] DATASETS...

Arguments:

• DATASETS...: The dataset(s) to use [required]

Options:

• -f, --force: Force deletion without confirmation
• --help: Show this message and exit.

evalio ls

List dataset and pipeline information

Usage:

$ evalio ls [OPTIONS] KIND:{datasets|pipelines}

Arguments:

• KIND:{datasets|pipelines}: The kind of object to list [required]

Options:

• -s, --search TEXT: Fuzzy search for a pipeline or dataset by name
• -q, --quiet: Output less verbose information
• --show-hyperlinks: Output full links. For terminals that don't support hyperlinks (OSC 8).
• --help: Show this message and exit.

evalio run

Run pipelines on datasets

Usage:

$ evalio run [OPTIONS]

Options:

• -c, --config PATH: Config file to load from
• -d, --datasets TEXT: The dataset(s) to use
• -p, --pipelines TEXT: The pipeline(s) to use
• -o, --output PATH: Output directory to save results
• -l, --length INTEGER: Number of scans to process for each dataset
• -v, --visualize: Visualize the results via rerun
• -s, --show PARSE: Show visualization options (m: map, i: image, s: scan, f: features). Automatically implies -v.
• --rerun-failed: Rerun failed experiments. If not set, will skip previously failed experiments.
• --help: Show this message and exit.

evalio stats

Evaluate the results of experiments.

Usage:

$ evalio stats [OPTIONS] DIRECTORIES...

Arguments:

• DIRECTORIES...: Directory of results to evaluate. [required]

Options:

• -v, --visualize: Visualize results.
• -S, --sort TEXT: Sort results by the name of a column. Defaults to RTEt.
• -r, --reverse: Reverse the sorting order. Defaults to False.
• -f, --filter TEXT: Python expressions to filter results rows. 'True' rows will be kept. Example: --filter 'RTEt < 0.5'
• --only-complete: Only show results for trajectories that completed.
• --only-failed: Only show results for trajectories that failed.
• -h, --hide TEXT: Columns to hide, may be repeated.
• -s, --show TEXT: Columns to force show, may be repeated.
• --print-columns: Print the names of all available columns.
• --w-meters FLOAT: Window size in meters for RTE computation. May be repeated. Defaults to 30m.
• --w-seconds FLOAT: Window size in seconds for RTE computation. May be repeated. Defaults to none.
• --metric [mean|median|sse]: Metric to use for ATE/RTE computation. Defaults to sse. [default: sse]
• -l, --length INTEGER: Specify subset of trajectory to evaluate.
• --help: Show this message and exit.