autopkgtest 4.0: Simplified CLI, deprecating “adt”

Historically, the “adt-run” command line has allowed multiple tests; as a consequence, arguments like --binary or --override-control were position dependent, which confused users a lot (#795274, #785068, #795274, LP #1453509). On the other hand I don’t know anyone or any CI system which actually makes use of the “multiple tests on a single command line” feature.

The command line also was a bit confusing in other ways, like the explicit --built-tree vs. --unbuilt-tree and the magic / vs. // suffixes, or option vs. positional arguments to specify tests.

The other long-standing confusion is the pervasive “adt” acronym, which is still from the very early times when “autopkgtest” was called “autodebtest” (this was changed one month after autodebtest’s inception, in 2006!).

Thus in some recent night/weekend hack sessions I’ve worked on a new command line interface and consistent naming. This is now available in autopkgtest 4.0 in Debian unstable and Ubuntu Yakkety. You can download and use the deb package on Debian jessie and Ubuntu ≥ 14.04 LTS as well. (I will provide official backports after the first bug fix release after this got some field testing.)

New “autopkgtest” command

The adt-run program is now superseded by autopkgtest:

  • It accepts only exactly one tested source package, and gives a proper error if none or more than one (often unintend) is given. Binaries to be tested, --override-control, etc. can now be specified in any order, making the arguments position independent. So you now can do things like: 

    autopkgtest *.dsc *.deb […]

    Before, *.deb only applied to the following test.

    • The explicit --source, --click-source etc. options are gone, the type of tested source/binary packages, including built vs. unbuilt tree, is detected automatically. Tests are now only specified with positional arguments, without the need (or possibility) to explicitly specify their type. The one exception is --installed-click com.example.myapp as possible names are the same as for apt source package names. 

      # Old:
adt-run –unbuilt-tree pkgs/foo-2 […]
# or equivalently:
adt-run pkgs/foo-2// […]
# New:
autopkgtest pkgs/foo-2

      # Old:
adt-run --git-source http://example.com/foo.git [...]
# New:
autopkgtest http://example.com/foo.git [...]

    • The virtualization server is now separated with a double instead of a tripe dash, as the former is standard Unix syntax.

    • It defaults to the current directory if that is a Debian source package. This makes the command line particularly simple for the common case of wanting to run tests in the package you are just changing: 

      autopkgtest – schroot sid

      Assuming the current directory is an unbuilt Debian package, this will build the package, and run the tests in ./debian/tests against the built binaries.

      • The virtualization server must be specified with its “short” name only, e. g. “ssh” instead of “adt-virt-ssh”. They also don’t get installed into $PATH any more, as it’s hardly useful to call them directly.

    README.running-tests got updated to the new CLI, as usual you can also read the HTML online.

    The old adt-run CLI is still available with unchanged behaviour, so it is safe to upgrade existing CI systems to that version.

    Image build tools

    All adt-build* tools got renamed to autopkgtest-build*, and got changed to build images prefixed with “autopkgtest” instead of “adt”. For example, adt-build-lxc ubuntu xenial now produces an autopkgtest-xenial container instead of adt-xenial.

    In order to not break existing CI systems, the new autopkgtest package contains symlinks to the old adt-build* commands, and when being called through them, also produce images with the old “adt-” prefix.

    Environment variables in tests

    Finally there is a set of environment variables that are exported by autopkgtest for using in tests and image customization tools, which now got renamed from ADT_* to AUTOPKGTEST_*:

    • AUTOPKGTEST_APT_PROXY
    • AUTOPKGTEST_ARTIFACTS
    • AUTOPKGTEST_AUTOPILOT_MODULE
    • AUTOPKGTEST_NORMAL_USER
    • AUTOPKGTEST_REBOOT_MARK
    • AUTOPKGTEST_TMP

    As these are being used in existing tests and tools, autopkgtest also exports/checks those under their old ADT_* name. So tests can be converted gradually over time (this might take several years).

    Feedback

    As usual, if you find a bug or have a suggestion how to improve the CLI, please file a bug in Debian or in Launchpad. The new CLI is recent enough that we still have some liberty to change it.

    Happy testing!

Home