Skip to content

evalio

Tool for evaluating Lidar-Inertial Odometry pipelines on open-source datasets

Usage:

$ evalio [OPTIONS] COMMAND [ARGS]...

Options:

  • -D, --data-dir PATH: Directory to store downloaded datasets.
  • -V, --version: Show version and exit.
  • --install-completion: Install completion for the current shell.
  • --show-completion: Show completion for the current shell, to copy it or customize the installation.
  • --help: Show this message and exit.

Commands:

  • dl: Download datasets
  • rm: Remove dataset(s)
  • ls: List dataset and pipeline information
  • run: Run pipelines on datasets
  • stats: Evaluate the results of experiments.

evalio dl

Download datasets

Usage:

$ evalio dl [OPTIONS] DATASETS...

Arguments:

  • DATASETS...: The dataset(s) to use [required]

Options:

  • --help: Show this message and exit.

evalio rm

Remove dataset(s)

If --force is not used, will ask for confirmation.

Usage:

$ evalio rm [OPTIONS] DATASETS...

Arguments:

  • DATASETS...: The dataset(s) to use [required]

Options:

  • -f, --force: Force deletion without confirmation
  • --help: Show this message and exit.

evalio ls

List dataset and pipeline information

Usage:

$ evalio ls [OPTIONS] KIND:{datasets|pipelines}

Arguments:

  • KIND:{datasets|pipelines}: The kind of object to list [required]

Options:

  • -s, --search TEXT: Fuzzy search for a pipeline or dataset by name
  • -q, --quiet: Output less verbose information
  • --show-hyperlinks: Output full links. For terminals that don't support hyperlinks (OSC 8).
  • --help: Show this message and exit.

evalio run

Run pipelines on datasets

Usage:

$ evalio run [OPTIONS]

Options:

  • -c, --config PATH: Config file to load from
  • -d, --datasets TEXT: The dataset(s) to use
  • -p, --pipelines TEXT: The pipeline(s) to use
  • -o, --output PATH: Output directory to save results
  • -l, --length INTEGER: Number of scans to process for each dataset
  • -v, --visualize: Visualize the results via rerun
  • -s, --show PARSE: Show visualization options (m: map, i: image, s: scan, f: features). Automatically implies -v.
  • --help: Show this message and exit.

evalio stats

Evaluate the results of experiments.

Usage:

$ evalio stats [OPTIONS] DIRECTORIES...

Arguments:

  • DIRECTORIES...: Directory of results to evaluate. [required]

Options:

  • -v, --visualize: Visualize results.
  • -s, --sort TEXT: Sort results by the name of a column. [default: RTEt]
  • -w, --window INTEGER: Window size for RTE. Defaults to 100 time-steps. [default: 200]
  • -m, --metric [mean|median|sse]: Metric to use for ATE/RTE computation. Defaults to sse. [default: sse]
  • -l, --length INTEGER: Specify subset of trajectory to evaluate.
  • --help: Show this message and exit.