↓ Skip to main content

πŸƒ Usage

The oinkie command-line utility provides all the subcommands required to perform the entire software birthmarking processβ€”from lifting binary files, extracting birthmarks, to comparing similarities.

The terms these pages use β€” birthmark, function, element, similarity and the rest β€” are each defined once in the glossary.


πŸš€ General Help and Subcommands
#

To view the basic usage and options of the oinkie CLI:

Detects software theft by comparing birthmarks extracted from binaries

Usage: oinkie [OPTIONS] <COMMAND>

Commands:
  info     Display information about the application
  lift     Lift binary files to JSON files of an intermediate representation, using a specified
           lifter
  extract  Extract birthmarks from a lifted binary file (JSON format)
  compare  Compare birthmarks and output the similarity score
  review   Re-read a finished comparison: recompute the similarity of each pair from the stored
           function similarities
  stats    Summarise a set of birthmarks: how many of each type, how many functions each holds, and
           how long each function's birthmark is
  run      Extract birthmarks and compare them in one command
  mcp      Serve oinkie over the Model Context Protocol, on stdin and stdout
  help     Print this message or the help of the given subcommand(s)

Options:
  -l, --level <LEVEL>  Log level for the application [default: warn] [possible values: error, warn,
                       info, debug, trace, off]
      --no-progress    Draw no progress bars, as when stderr is kept in a log or a CI output
  -h, --help           Print help
  -V, --version        Print version

πŸƒ Steps in detail
#

The toolkit’s operations are divided into distinct stages. You can execute them individually to examine results at each stage, or run them all at once with the run command:

  1. Lifting Binaries to Pcode (OIR)
    Convert your raw executable or compiled binary files into the Oinkie Intermediate Representation (OIR) JSON format using Ghidra.

  2. Extracting Birthmarks
    Analyze the generated OIR JSON files to extract specific birthmarks based on opcodes, function calls, or \(k\)-grams.

  3. Comparing Birthmarks
    Compare extracted birthmarks between pairs of files using chosen similarity algorithms and matching heuristics.

  4. Reviewing Scores
    Recalculate the similarity of each pair from saved function similarities.

  5. All-in-One Execution (Run)
    Execute extraction and comparison together in a single command.


βš™οΈ Threads, memory and time
#

Comparing is where the resources go, and how much depends on the pairs:

  • Memory. Comparing a pair holds one number for every pair of their functions, 8 bytes each: two birthmarks of 5,000 functions take 200 MB, two of 50,000 take 20 GB. Several pairs are compared at once, so the largest pairs and the number of threads together set the peak.
  • Time. It grows with the same count of function pairs, and with the algorithm: lcs and levenshtein work through both functions’ sequences for every pair of functions, the set and frequency algorithms only through what the two hold. On a 10-core machine, two birthmarks of 5,000 functions compare in a few seconds under jaccard and in under a minute under lcs. The hungarian aggregator adds its own share on large pairs; topn:N adds almost none.
  • Threads. compare, run, review, extract, stats and mcp compute on one thread per core by default. --threads N bounds all of it to \(N\) threads – the pairs, the files, and the rows of each pair’s matrix share them – and lowers memory with it. Nothing else sets the number; an environment variable such as RAYON_NUM_THREADS has no effect.
  • lift --jobs is a different knob. It counts decompiler processes, each a whole Ghidra, Binary Ninja or IDA run, and is bounded by their memory rather than by cores. Lifting is serial unless --jobs says otherwise; see lift.

Summarising Birthmarks (`stats` command)

·617 words
The stats command summarises a set of extracted birthmarks: how many there are of each type, how many functions each program holds, and how long each function’s birthmark is. It is what you would otherwise write a script for when describing a dataset in a paper, or when choosing a threshold on birthmark length. πŸ“ Functions and elements # A birthmark file is one program, and it holds one entry per function. Each function’s birthmark is made of elements: operations, calls or \(k\)-grams. The two counts are kept apart by name:

Serving to an Agent (`mcp` command)

·744 words
The mcp command serves the birthmark pipeline over the Model Context Protocol, on stdin and stdout. An agent can then ask whether two programs are alike without a person chaining extract and compare by hand. The subcommand is behind a cargo feature. The released binaries and both container images carry it; building from source needs it asked for: cargo build --release --features mcp βš™οΈ Configuring a client # .mcp.json, in a project directory:

Application Information (`info` command)

·605 words
The info command displays general details about the oinkie application, including the version, author, and all currently supported birthmark configurations and comparison algorithms. πŸƒ Usage # oinkie info [OPTIONS] Options # -h, --help Print help information. πŸ“Š Example Output # Running the info command provides detailed listings of supported formats and algorithms in your current environment:

1. Lifting Binaries (`lift` command)

·1099 words
The lift command converts compiled binary executable files into the Oinkie Intermediate Representation (OIR) JSON format. It drives Ghidra in headless mode, producing its P-Code, or Binary Ninja, producing any of its three intermediate languages. --ir chooses which, and the choice is recorded in the file, because the two vocabularies are not interchangeable. The examples below use Ghidra’s P-Code. For Binary Ninja, see the lifter’s own notes. πŸƒ Usage # You can lift multiple binary files at once. The default behavior is to use Ghidra to analyze the executables and output the resulting JSON files to the ./pcodes directory.

2. Extracting Birthmarks (`extract` command)

·359 words
The extract command analyzes the lifted OIR (JSON) files and extracts specified types of birthmarks. A software birthmark represents a distinct structural or semantic characteristic of a program (like instruction types or external function dependencies) that remains stable across compilation variations. πŸƒ Usage # oinkie extract [OPTIONS] [FILES]... Arguments # <FILES>... Path to the OIR JSON files (previously generated via the lift command) from which to extract birthmarks. Options # -d, --dest <DIRECTORY> Specify the destination directory for saving the extracted birthmark files. Defaults to the ./birthmarks directory. [default: birthmarks] -b, --birthmark-type <BIRTHMARK_TYPE> The type and representation structure of the birthmark to extract. Defaults to op-seq. [default: op-seq] Refer below for a full list of supported birthmark types. -S, --skip Skip the extraction process if the output birthmark file already exists. --threads <N> The number of threads to compute on, for every part of the command that runs in parallel. [default: one per core] πŸ§ͺ Birthmark Types and Structural Representation # Software birthmarks are categorized by their underlying element types and their representation structures.

3. Comparing Birthmarks (`compare` command)

·1127 words
The compare command performs pairwise comparison between extracted birthmarks to measure their similarity. This step calculates a score between \(0.0\) (entirely different) and \(1.0\) (identical), indicating the level of structural similarity between the programs. πŸƒ Usage # oinkie compare [OPTIONS] [JSON_FILES]... Arguments # <JSON_FILES>... Paths to the birthmark JSON files (generated using the extract command) to compare. Options # -a, --algorithm <ALGORITHM> Specify the similarity calculation algorithm to compare birthmarks. [default: jaccard] Refer below for a full list of algorithms. -A, --aggregator <METHOD> Specify the method for combining the function similarities of a pair into its single similarity. [default: hungarian] Refer below for a full list of aggregators. -s, --strategy <STRATEGY> The pairing strategy to use when comparing files. [default: all-and-self] [possible values: all-and-self, all, self-coverage, adjacent, first-vs-others, last-vs-others] Refer below for a full list of strategies. -d, --dest <DIRECTORY> The output directory where comparison results are saved (typically as CSV files). [default: similarities] -S, --skip Skip the comparison if the output file already exists for the current file pair. --threads <N> The number of threads to compute on, for every part of the command that runs in parallel. [default: one per core] 🧦 Pairing Strategies (--strategy) # When comparing multiple files, you can configure which file pairs are compared using the following strategy options:

4. Reviewing Scores (`review` command)

·623 words
The review command re-reads a comparison that is already finished. It recalculates the similarity of each pair from the function similarities a comparison stored. This avoids having to completely rerun expensive pairwise comparisons if you simply want to test a different aggregation method (e.g., swapping between hungarian and topn:N). review was called reaggregate before v0.8.0. The old name is refused with a message naming review. πŸƒ Usage # oinkie review [OPTIONS] <SCORE_DIRECTORY> Arguments # <SCORE_DIRECTORY> Path to the score directory, which holds the saved function similarities (often located inside the output folder generated by the compare command). Options # -A, --aggregator <METHOD> Specify the aggregator method to combine the function similarities of a pair into its single similarity. [default: hungarian] hungarian: Optimal overall bipartite matching between functions. topn:N: Average the \(N\) best of each function’s best matches, from each side. containment: The hungarian pairing, averaged over the smaller side’s functions: how much of the smaller is found in the larger. matched:T: The proportion of functions the hungarian pairing matches with a similarity of at least \(T\). weighted: A one-to-one pairing in which each function counts by its number of elements. It reads the sizes from the birthmarks the comparison names, as --min-elements does, so those files must still be there and unchanged. -d, --dest-file <RESULT.CSV> The path to the destination CSV file where the recomputed similarity of each pair is saved. [default: review.csv] --min-elements <N|Rx> Drop the functions with fewer elements than this from each pair before aggregating. See Dropping short functions. N: a count of elements, such as 5. Rx: (R) times the mean element count, such as 0.3x. A bare 0.3 is refused. --threads <N> The number of threads to compute on, for every part of the command that runs in parallel. [default: one per core] πŸ’‘ Practical Workflow Example # Imagine you have run a large comparison across hundreds of files using the standard hungarian aggregator:

5. All-in-One Execution (`run` command)

·441 words
The run command is an all-in-one command designed to perform both extraction and comparison of software birthmarks in a single transaction. This is ideal for quick checks or automated CI pipelines, saving the intermediate step of writing extracted birthmarks manually to separate folders. πŸƒ Usage # oinkie run [OPTIONS] [FILES]... Arguments # <FILES>... Paths to the OIR JSON files (previously generated via the lift command) to extract and compare. Options # -a, --analysis <ANALYSIS> The combination of birthmark-type, representation, and similarity-algorithm to evaluate. [default: op-set-jaccard] Refer below for a list of common configuration formats. -A, --aggregator <METHOD> Specify the method for combining the function similarities of a pair into its single similarity. [default: hungarian] Refer below for a full list of aggregators. -s, --strategy <STRATEGY> The pairing strategy to use when comparing files. [default: all-and-self] [possible values: all-and-self, all, self-coverage, adjacent, first-vs-others, last-vs-others] Refer a full list in compare subcommand. -d, --dest <DIRECTORY> Destination directory for output CSV files containing the similarity results. [default: similarities] The birthmarks the comparisons were made from are written into its birthmarks/ subdirectory, one per input, and each pair’s CSV names the two it used – so the directory holds everything review needs to re-read the comparison. -S, --skip Reuse what an earlier run left in the destination. A birthmark already in birthmarks/ is reused if it is of the analysis’s birthmark type, and extracted again otherwise. A pair’s CSV is reused only if both birthmarks it was computed from were reused; otherwise the pair is compared again, so a CSV never names a birthmark it was not computed from. --threads <N> The number of threads to compute on, for every part of the command that runs in parallel. [default: one per core] πŸ”¬ Specifying Analysis Modes (--analysis) # The format for --analysis strings is structured as: