As discussed with tfc before, it's probably better in most cases to try
the interactive test-driver with the SSH backdoor first and resort to
this feature for flaky tests.
Without a proper introduction it's really really hard to make sense of the examples, and where values come from; which are arbitrary, which are conventional, which are hard-coded into some part of the framework.
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
Right now it wrongly seems as if you can set
`sshBackdoor.enable = true;` for each test and not only for debugging
purposes.
This is wrong however since you'd need to pass /dev/vhost-vsock into the
sandbox for this (which is also a prerequisite for #392117).
To make that clear, two things were changed:
* add a warning to the manual to communicate this.
* exit both interactive and non-interactive driver early if
/dev/vhost-vsock is missing and the ssh backdoor is enabled.
If that's the case, we pass a CLI flag to the driver already in the
interactive case. This change also sets the flag for the
non-interactive case.
That way we also get a better error if somebody tries to enable this
on a system that doesn't support that.
I'm a little annoyed at myself that I only realized this _after_ #392030
got merged. But I realized that if something else is using AF_VSOCK or
you simply have another interactive test running (e.g. by another user
on a larger builder), starting up VMs in the driver fails with
qemu-system-x86_64: -device vhost-vsock-pci,guest-cid=3: vhost-vsock: unable to set guest cid: Address already in use
Multi-user setups are broken anyways because you usually don't have
permissions to remove the VM state from another user and thus starting
the driver fails with
PermissionError: [Errno 13] Permission denied: PosixPath('/tmp/vm-state-machine')
but this is something you can work around at least.
I was considering to generate random offsets, but that's not feasible
given we need to know the numbers at eval time to inject them into the
QEMU args. Also, while we could do this via the test-driver, we should
also probe if the vsock numbers are unused making the code even more
complex for a use-case I consider rather uncommon.
Hence the solution is to do
sshBackdoor.vsockOffset = 23542;
when encountering conflicts.
With this it's possible to trivially SSH into running machines from the
test-driver. This is especially useful when running VM tests
interactively on a remote system.
This is based on `systemd-ssh-proxy(1)`, so there's no need to configure
any additional networking on the host-side.
Suggested-by: Ryan Lahfa <masterancpp@gmail.com>
Replaces / Closes#345948
I tried to integrate `pytest` assertions because I like the reporting,
but I only managed to get the very basic thing and even that was messing
around a lot with its internals.
The approach in #345948 shifts too much maintenance effort to us, so
it's not really desirable either.
After discussing with Benoit on Ocean Sprint about this, we decided that
it's probably the best compromise to integrate `unittest`: it also
provides good diffs when needed, but the downside is that existing tests
don't benefit from it.
This patch essentially does the following things:
* Add a new global `t` that is an instance of a `unittest.TestCase`
class. I decided to just go for `t` given that e.g.
`tester.assertEqual` (or any other longer name) seems quite verbose.
* Use a special class for errors that get special treatment:
* The traceback is minimized to only include frames from the
testScript: in this case I don't really care about anything else and
IMHO that's just visual noise.
This is not the case for other exceptions since these may indicate a
bug and then people should be able to send the full traceback to the
maintainers.
* Display the error, but with `!!!` as prefix to make sure it's
easier to spot in between other logs.
This looks e.g. like
!!! Traceback (most recent call last):
!!! File "<string>", line 7, in <module>
!!! foo()
!!! File "<string>", line 5, in foo
!!! t.assertEqual({"foo":[1,2,{"foo":"bar"}]},{"foo":[1,2,{"bar":"foo"}],"bar":[1,2,3,4,"foo"]})
!!!
!!! NixOSAssertionError: {'foo': [1, 2, {'foo': 'bar'}]} != {'foo': [1, 2, {'bar': 'foo'}], 'bar': [1, 2, 3, 4, 'foo']}
!!! - {'foo': [1, 2, {'foo': 'bar'}]}
!!! + {'bar': [1, 2, 3, 4, 'foo'], 'foo': [1, 2, {'bar': 'foo'}]}
cleanup
kill machine (pid 9)
qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/wz0j2zi02rvnjiz37nn28h3gfdq61svz-python3-3.12.9/bin/python3.12)
kill vlan (pid 7)
(finished: cleanup, in 0.00 seconds)
Co-authored-by: bew <bew@users.noreply.github.com>
This gives people some flexibility when they need a path type, and
prevents a "combinatorial explosion" of various path stops.
I've re-implemented our existing `path` and `pathInStore` types using
`pathWith`. Our existing `package` type is potentially a candidate for
similar treatment, but it's a little quirkier (there's some stuff with
`builtins.hasContext` and `toDerivation` that I don't completely
understand), and I didn't want to muddy this PR with that.
As a happy side effect of this work, we get a new feature: the ability
to create a type for paths *not* in the store. This is useful for when a
module needs a path to a file, and wants to protect people from
accidentally leaking that file into the nix store.
