CLI#

Bowtie is a versatile tool which you can use to investigate any or all of the implementations it supports. Below are a few sample command lines you might be interested in.

Running Commands Against All Implementations

Many of Bowtie’s subcommands take a -i / --implementation option which specifies which implementations you wish to run against. In general, these same subcommands allow repeating this argument multiple times to run across multiple implementations. In many or even most cases, you may be interested in running against all implementations Bowtie supports. For somewhat boring reasons (partially having to do with the GitHub API) this “run against all implementations” functionality is slightly nontrivial to implement in a seamless way, though doing so is nevertheless tracked in this issue.

In the interim, it’s often convenient to use a local checkout of Bowtie in order to list this information.

Specifically, all supported implementations live in the implementations/ directory, and therefore you can construct a string of -i arguments using a small bit of shell vomit. If you have cloned Bowtie to /path/to/bowtie you should be able to use $(ls /path/to/bowtie/implementations/ | sed 's/^| /-i /') in any command to expand out to all implementations. See below for a full example.

Validating a Specific Instance Against One or More Implementations#

The bowtie validate subcommand can be used to test arbitrary schemas and instances against any implementation Bowtie supports.

Given some collection of implementations to check – here perhaps two Javascript implementations – it takes a single schema and one or more instances to check against it:

$ bowtie validate -i js-ajv -i js-hyperjump <(printf '{"type": "integer"}') <(printf 37) <(printf '"foo"')

Note that the schema and instance arguments are expected to be files, and that therefore the above makes use of normal shell process substitution to pass some examples on the command line.

Piping this output to bowtie summary is often the intended outcome (though not always, as you also may upload the output it gives to https://bowtie.report/ as a local report). For summarizing the results in the terminal however, the above command when summarized produces:

$ bowtie validate -i js-ajv -i js-hyperjump <(printf '{"type": "integer"}') <(printf 37) <(printf '"foo"') | bowtie summary
2023-11-02 15:43.10 [debug    ] Will speak                     dialect=https://json-schema.org/draft/2020-12/schema
2023-11-02 15:43.10 [info     ] Finished                       count=1
                                        Bowtie
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Schema              ┃                                                              ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│                     │                                                              │
│ {                   │   Instance   ajv (javascript)   hyperjump-jsv (javascript)   │
│   "type": "integer" │  ──────────────────────────────────────────────────────────  │
│ }                   │   37         valid              valid                        │
│                     │   "foo"      invalid            invalid                      │
│                     │                                                              │
└─────────────────────┴──────────────────────────────────────────────────────────────┘
                                    2 tests ran

Running a Single Test Suite File#

To run the draft 7 type-keyword tests on the Lua jsonschema implementation, run:

$ bowtie suite -i lua-jsonschema https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/type.json | bowtie summary --show failures

Running the Official Suite Across All Implementations#

The following will run all Draft 7 tests from the official test suite (which it will automatically retrieve) across all implementations supporting Draft 7, and generate an HTML report named bowtie-report.html in the current directory:

$ bowtie suite $(ls /path/to/bowtie/implementations/ | sed 's/^| /-i /') https://github.com/json-schema-org/JSON-Schema-Test-Suite/tree/main/tests/draft7 | bowtie summary --show failures

Running Test Suite Tests From Local Checkouts#

Providing a local path to the test suite can be used as well, which is useful if you have local changes:

$ bowtie suite $(ls /path/to/bowtie/implementations/ | sed 's/^| /-i /') ~/path/to/json-schema-org/suite/tests/draft2020-12/ | bowtie summary --show failures

Checking An Implementation Functions On Basic Input#

If you wish to verify that a particular implementation works on your machine (e.g. if you suspect a problem with the container image, or otherwise aren’t seeing results), you can run bowtie smoke. E.g., to verify the Golang jsonschema implementation is functioning, you can run:

$ bowtie smoke -i go-jsonschema

Reference#

bowtie#

A meta-validator for the JSON Schema specifications.

bowtie [OPTIONS] COMMAND [ARGS]...

Options

--version#

Show the version and exit.

badges#

Generate Bowtie badges from a previous run.

bowtie badges [OPTIONS] OUTPUT

Options

--input <input>#

Arguments

OUTPUT#

Required argument

info#

Retrieve a particular implementation (harness)’s metadata.

bowtie info [OPTIONS]

Options

-f, --format <format>#

What format to use for the output

Default:

pretty if stdout is a tty, otherwise JSON

Options:

json | pretty

-i, --implementation <image_names>#

Required A docker image which implements the bowtie IO protocol.

run#

Run a sequence of cases provided on standard input.

bowtie run [OPTIONS] [INPUT]

Options

-i, --implementation <image_names>#

Required A docker image which implements the bowtie IO protocol.

-D, --dialect <dialect>#

A URI or shortname identifying the dialect of each test case.Shortnames include: [‘2019’, ‘2019-09’, ‘201909’, ‘2020’, ‘2020-12’, ‘202012’, ‘3’, ‘4’, ‘6’, ‘7’, ‘draft2019-09’, ‘draft201909’, ‘draft2020-12’, ‘draft202012’, ‘draft3’, ‘draft4’, ‘draft6’, ‘draft7’].

Default:

draft2020-12

-k <filter>#

Only run cases whose description match the given glob pattern.

-x, --fail-fast#

Fail immediately after the first error or disagreement.

-S, --set-schema, --no-set-schema#

Explicitly set $schema in all (non-boolean) case schemas sent to implementations. Note this of course means what is passed to implementations will differ from what is provided in the input.

Default:

False

-T, --read-timeout <SECONDS>#

An explicit timeout to wait for each implementation to respond to each instance being validated. Set this to 0 if you wish to wait forever, though note that this means you may end up waiting … forever!

Default:

2.0

-V, --validate-implementations#

When speaking to implementations (provided via -i), validate the requests and responses sent to them under Bowtie’s JSON Schema specification. Generally, this option protects against broken Bowtie implementations and can be left at its default (of off) unless you are developing a new implementation container.

Arguments

INPUT#

Optional argument

smoke#

Smoke test one or more implementations for basic correctness.

bowtie smoke [OPTIONS]

Options

-q, --quiet#

Don’t print any output, just exit with nonzero status on failure.

-f, --format <format>#

What format to use for the output

Default:

pretty if stdout is a tty, otherwise JSON

Options:

json | pretty

-i, --implementation <image_names>#

Required A docker image which implements the bowtie IO protocol.

suite#

Run test cases from the official JSON Schema test suite.

Supports a number of possible inputs:

  • file paths found on the local file system containing tests, e.g.:

    • {PATH}/tests/draft7 to run the draft 7 version’s tests out of a local checkout of the test suite

    • {PATH}/tests/draft7/foo.json to run just one file from a checkout

  • URLs to the test suite repository hosted on GitHub, e.g.:

    • https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/ to run a version directly from any branch which exists in GitHub

    • https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/foo.json to run a single file directly from a branch which exists in GitHub

  • short name versions of the previous URLs (similar to those providable to bowtie validate via its --dialect option), e.g.:

    • 7, to run the draft 7 tests directly from GitHub (as in the URL example above)

bowtie suite [OPTIONS] INPUT

Options

-i, --implementation <image_names>#

Required A docker image which implements the bowtie IO protocol.

-k <filter>#

Only run cases whose description match the given glob pattern.

-x, --fail-fast#

Fail immediately after the first error or disagreement.

-S, --set-schema, --no-set-schema#

Explicitly set $schema in all (non-boolean) case schemas sent to implementations. Note this of course means what is passed to implementations will differ from what is provided in the input.

Default:

False

-T, --read-timeout <SECONDS>#

An explicit timeout to wait for each implementation to respond to each instance being validated. Set this to 0 if you wish to wait forever, though note that this means you may end up waiting … forever!

Default:

2.0

-V, --validate-implementations#

When speaking to implementations (provided via -i), validate the requests and responses sent to them under Bowtie’s JSON Schema specification. Generally, this option protects against broken Bowtie implementations and can be left at its default (of off) unless you are developing a new implementation container.

Arguments

INPUT#

Required argument

summary#

Generate an (in-terminal) summary of a Bowtie run.

bowtie summary [OPTIONS] [INPUT]

Options

-f, --format <format>#

What format to use for the output

Default:

pretty if stdout is a tty, otherwise JSON

Options:

json | pretty

-s, --show <show>#

Configure whether to display validation results (whether instances are valid or not) or test failure results (whether the validation results match expected validation results)

Default:

validation

Options:

failures | validation

Arguments

INPUT#

Optional argument

tui#

Open a simple interactive TUI for executing Bowtie commands.

bowtie tui [OPTIONS]
validate#

Validate one or more instances under a given schema across implementations.

bowtie validate [OPTIONS] SCHEMA [INSTANCES]...

Options

-i, --implementation <image_names>#

Required A docker image which implements the bowtie IO protocol.

-D, --dialect <dialect>#

A URI or shortname identifying the dialect of each test case.Shortnames include: [‘2019’, ‘2019-09’, ‘201909’, ‘2020’, ‘2020-12’, ‘202012’, ‘3’, ‘4’, ‘6’, ‘7’, ‘draft2019-09’, ‘draft201909’, ‘draft2020-12’, ‘draft202012’, ‘draft3’, ‘draft4’, ‘draft6’, ‘draft7’].

Default:

draft2020-12

-S, --set-schema, --no-set-schema#

Explicitly set $schema in all (non-boolean) case schemas sent to implementations. Note this of course means what is passed to implementations will differ from what is provided in the input.

Default:

False

-T, --read-timeout <SECONDS>#

An explicit timeout to wait for each implementation to respond to each instance being validated. Set this to 0 if you wish to wait forever, though note that this means you may end up waiting … forever!

Default:

2.0

-V, --validate-implementations#

When speaking to implementations (provided via -i), validate the requests and responses sent to them under Bowtie’s JSON Schema specification. Generally, this option protects against broken Bowtie implementations and can be left at its default (of off) unless you are developing a new implementation container.

--expect <expect>#

Expect the given input to be considered valid or invalid, or else (with ‘any’) to allow either result.

Default:

any

Options:

valid | invalid | any

Arguments

SCHEMA#

Required argument

INSTANCES#

Optional argument(s)