Commit Graph
100 Commits
Author SHA1 Message Date
Daniel P. Berrangé aa5730b07e tests/docker: replace fedora-i386 with debian-i686
Fedora is gradually killing off i386 packages in its repos, via a
death-by-1000-cuts process. Thus Debian looks like a better long
term bet for i686 build testing. It has the added advantage that
we can generate it via lcitool too.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20231107164109.1449014-1-berrange@redhat.com>
[AJB: tweak commit msg, set correct prefix]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-2-alex.bennee@linaro.org>
2023-11-23 11:55:32 +00:00
Daniel P. Berrangé cc9118f240 MAINTAINERS: update libvirt devel mailing list address
Effective immediately, the libvirt project has moved its list off
libvir-list@redhat.com, to devel@lists.libvirt.org

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-07 15:18:36 +00:00
Daniel P. Berrangé 5b4edd7230 audio: don't abort on f32 audio format in wav backend
Print a debug message as is done for other unsupported audio formats
to give the user the chance to understand their mistake.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-07 14:58:54 +00:00
Daniel P. Berrangé f5c5e7d9db MAINTAINERS: update libvirt devel mailing list address
Effective immediately, the libvirt project has moved its list off
libvir-list@redhat.com, to devel@lists.libvirt.org

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20231027095643.2842382-1-berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-11-07 13:08:49 +01:00
Daniel P. Berrangé e307a8174b qapi: provide a friendly string representation of QAPI classes
If printing a QAPI schema object for debugging we get the classname and
a hex value for the instance:

  <qapi.schema.QAPISchemaEnumType object at 0x7f0ab4c2dad0>
  <qapi.schema.QAPISchemaObjectType object at 0x7f0ab4c2dd90>
  <qapi.schema.QAPISchemaArrayType object at 0x7f0ab4c2df90>

With this change we instead get the classname and the human friendly
name of the QAPI type instance:

  <QAPISchemaEnumType:CpuS390State at 0x7f0ab4c2dad0>
  <QAPISchemaObjectType:CpuInfoS390 at 0x7f0ab4c2dd90>
  <QAPISchemaArrayType:CpuInfoFastList at 0x7f0ab4c2df90>

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20231018120500.2028642-1-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Conditional swapped to avoid negation]
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
[Tweaked to mollify pylint]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2023-10-19 07:21:37 +02:00
Daniel P. Berrangé 9afa888ce0 osdep: set _FORTIFY_SOURCE=2 when optimization is enabled
Currently we set _FORTIFY_SOURCE=2 as a compiler argument when the
meson 'optimization' setting is non-zero, the compiler is GCC and
the target is Linux.

While the default QEMU optimization level is 2, user could override
this by setting CFLAGS="-O0" or --extra-cflags="-O0" when running
configure and this won't be reflected in the meson 'optimization'
setting. As a result we try to enable _FORTIFY_SOURCE=2 and then the
user gets compile errors as it only works with optimization.

Rather than trying to improve detection in meson, it is simpler to
just check the __OPTIMIZE__ define from osdep.h.

The comment about being incompatible with clang appears to be
outdated, as compilation works fine without excluding clang.

In the coroutine code we must set _FORTIFY_SOURCE=0 to stop the
logic in osdep.h then enabling it.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20231003091549.223020-1-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-04 09:52:06 -04:00
Daniel P. Berrangé 0d57919acf seccomp: avoid shadowing of 'action' variable
This is confusing as one 'action' variable is used for storing
a SCMP_ enum value, while the other 'action' variable is used
for storing a SECCOMP_ enum value.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20230922160644.438631-3-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2023-09-29 10:07:20 +02:00
Daniel P. Berrangé 3cc9fe177f crypto: remove shadowed 'ret' variable
Both instances of 'ret' are used to store a gnutls API return code.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20230922160644.438631-2-berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2023-09-29 10:07:20 +02:00
Daniel P. Berrangé eca74afd7d gitlab: make Cirrus CI jobs gating
The Cirrus CI jobs have been non-gating for a while to let us build
confidence in their reliability. Aside from periodic dependancy
problems when FreeBSD Ports switches to be based on a new FreeBSD
image version, the jobs have been reliable. It is thus worth making
them gating to prevent build failures being missed during merges.

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-5-berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-8-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Daniel P. Berrangé c576d8bfbb gitlab: make Cirrus CI timeout explicit
On the GitLab side we're invoking the Cirrus CI job using the
cirrus-run tool which speaks to the Cirrus REST API. Cirrus
sometimes tasks 5-10 minutes to actually schedule the task,
and thus the execution time of 'cirrus-run' inside GitLab will
be slightly longer than the execution time of the Cirrus CI
task.

Setting the timeout in the GitLab CI job should thus be done
in relation to the timeout set for the Cirrus CI job. While
Cirrus CI defaults to 60 minutes, it is better to set this
explicitly, and make the relationship between the jobs
explicit

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-4-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-7-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Daniel P. Berrangé 49e9f8644b qtest: kill orphaned qtest QEMU processes on FreeBSD
On Linux we use PR_SET_PDEATHSIG to kill orphaned QEMU processes
if we fail to call qtest_quit(), or the test program aborts/segvs.
This prevents meson from hanging forever due to the orphaned
process keeping stdout open.

On FreeBSD we can achieve the same using PROC_PDEATHSIG_CTL, which
gives us the equivalent protection against hangs.

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-6-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Daniel P. Berrangé 81395b6ec8 microbit: add missing qtest_quit() call
Without this call, the QEMU process is being left running which on
FreeBSD 13.2 at least, makes meson think the test is still running,
and thus execution of "make check" continues forever.

This fixes the regression introduced in:

  commit a9c9bbee85
  Author: Chris Laplante <chris@laplante.io>
  Date:   Tue Aug 22 17:31:02 2023 +0100

    qtest: microbit-test: add tests for nRF51 DETECT

Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1882
Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-5-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Daniel P. Berrangé 2f7350cd43 gitlab: enable ccache for many build jobs
The `ccache` tool can be very effective at reducing compilation times
when re-running pipelines with only minor changes each time. For example
a fresh 'build-system-fedora' job will typically take 20 minutes on the
gitlab.com shared runners. With ccache this is reduced to as little as
6 minutes.

Normally meson would auto-detect existance of ccache in $PATH and use
it automatically, but the way we wrap meson from configure breaks this,
as we're passing in an config file with explicitly set compiler paths.
Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
unknown reasons if doing this in msys though, gcc becomes unable to
invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
before invoking 'configure' instead.

A second problem with msys is that cache misses are incredibly
expensive, so enabling ccache massively slows down the build when
the cache isn't well populated. This is suspected to be a result of
the cost of spawning processes under the msys architecture. To deal
with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
strategy. This avoids extra spawning of the pre-processor during
cache misses, with the downside that is it less likely ccache will
find a cache hit after semantically benign compiler flag changes.
This is the lesser of two evils, as otherwise we can't use ccache
at all under msys and remain inside the job time limit.

If people are finding ccache to hurt their pipelines, it can be
disabled by setting the 'CCACHE_DISABLE=1' env variable against
their gitlab fork CI settings.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230804111054.281802-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-30 14:57:33 +01:00
Daniel P. Berrangé f54ba56dad gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
The FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from
each line of the job script to be presented in a collapsible section
with execution time listed.

While it works on Linux shared runners, when used with Windows runners
with PowerShell, this option does not create any sections, and actually
causes echo'ing of commands to be disabled, making it even worse to
debug the jobs.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-9-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé 63f5365cd4 gitlab: disable optimization and debug symbols in msys build
Building at -O2, adds 33% to the build time, over -O2. IOW a build that
takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug
symbols drops it further, down to 38 minutes.

IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the
gitlab CI windows shared runners.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230801130403.164060-8-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé ff136d2a99 configure: support passthrough of -Dxxx args to meson
This can be useful for setting some meson global options, such as the
optimization level or debug state.xs

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230801130403.164060-7-berrange@redhat.com>
[thuth: Move the help text into the section with the other --... options]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé ef4fe31f33 gitlab: always populate cache for windows msys jobs
The cache is used to hold the msys installer. Even if the build phase
fails, we should still populate the cache as the installer will be
valid for next time.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-6-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 5ef56e3b18 gitlab: drop $CI_PROJECT_DIR from cache path
The gitlab cache is limited to only handle content within the
$CI_PROJECT_DIR hierarchy, and as such relative paths are always
implicitly relative to $CI_PROJECT_DIR.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 46aedd34b4 gitlab: always use updated msys installer
We current reference an msys installer binary from mid-2022, which means
after installation, it immediately has to re-download a bunch of newer
content. This wastes precious CI time.

The msys project publishes an installer binary with a fixed URL that
always references the latest content. We cache the downloads in gitlab
though and so once downloaded we would never re-fetch the installer
leading back to the same problem.

To deal with this we also fetch the pgp signature for the installer
on every run, and compare that to the previously cached signature. If
the signature changes, we re-download the full installer.

This ensures we always have the latest installer for msys, while also
maximising use of the gitlab cache.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé b64052cdad gitlab: print timestamps during windows msys jobs
It is hard to get visibility into where time is consumed in our Windows
msys jobs. Adding a few log console messages with the timestamp will
aid in our debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 11961d08fc gitlab: remove duplication between msys jobs
Although they share a common parent, the two msys jobs still have
massive duplication in their script definitions that can easily be
collapsed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 10be627d2b io: remove io watch if TLS channel is closed during handshake
The TLS handshake make take some time to complete, during which time an
I/O watch might be registered with the main loop. If the owner of the
I/O channel invokes qio_channel_close() while the handshake is waiting
to continue the I/O watch must be removed. Failing to remove it will
later trigger the completion callback which the owner is not expecting
to receive. In the case of the VNC server, this results in a SEGV as
vnc_disconnect_start() tries to shutdown a client connection that is
already gone / NULL.

CVE-2023-3354
Reported-by: jiangyegen <jiangyegen@huawei.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-08-01 18:45:27 +01:00
Daniel P. Berrangé e02f56e3de tests/qtest: massively speed up migration-test
The migration test cases that actually exercise live migration want to
ensure there is a minimum of two iterations of pre-copy, in order to
exercise the dirty tracking code.

Historically we've queried the migration status, looking for the
'dirty-sync-count' value to increment to track iterations. This was
not entirely reliable because often all the data would get transferred
quickly enough that the migration would finish before we wanted it
to. So we massively dropped the bandwidth and max downtime to
guarantee non-convergance. This had the unfortunate side effect
that every migration took at least 30 seconds to run (100 MB of
dirty pages / 3 MB/sec).

This optimization takes a different approach to ensuring that a
mimimum of two iterations. Rather than waiting for dirty-sync-count
to increment, directly look for an indication that the source VM
has dirtied RAM that has already been transferred.

On the source VM a magic marker is written just after the 3 MB
offset. The destination VM is now montiored to detect when the
magic marker is transferred. This gives a guarantee that the
first 3 MB of memory have been transferred. Now the source VM
memory is monitored at exactly the 3MB offset until we observe
a flip in its value. This gives us a guaranteed that the guest
workload has dirtied a byte that has already been transferred.

Since we're looking at a place that is only 3 MB from the start
of memory, with the 3 MB/sec bandwidth, this test should complete
in 1 second, instead of 30 seconds.

Once we've proved there is some dirty memory, migration can be
set back to full speed for the remainder of the 1st iteration,
and the entire of the second iteration at which point migration
should be complete.

On a test machine this further reduces the migration test time
from 8 minutes to 1 minute 40.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-11-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-07-10 15:34:57 +02:00
Daniel P. Berrangé dcb242c849 gitlab: ensure coverage job also publishes meson log
The coverage job wants to publish a coverage report on success, but the
tests might fail and in that case we need the meson logs for debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503145535.91325-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-3-alex.bennee@linaro.org>
2023-07-03 12:51:08 +01:00
Daniel P. Berrangé cef63308e6 gitlab: explicit set artifacts publishing criteria
If not set explicitly, gitlab assumes 'when: on_success" as the
publishing criteria for artifacts. This is reasonable if the
artifact is an output deliverable of the job. This is useless
if the artifact is a log file to be used for debugging job
failures.

This change makes the desired criteria explicit for every job
that publishes artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503145535.91325-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-2-alex.bennee@linaro.org>
2023-07-03 12:51:01 +01:00
Daniel P. Berrangé 49ac76c2a0 gitlab: support disabling job auto-run in upstream
In forks QEMU_CI=1 can be used to create a pipeline but not auto-run any
jobs. In upstream jobs always auto-run, which is equiv of QEMU_CI=2.

This supports setting QEMU_CI=1 in upstream, to disable job auto-run.
This can be used to preserve CI minutes if repushing a branch to staging
with a specific fix that only needs testing in limited scenarios.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-6-berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé a77ef83cf8 gitlab: avoid extra pipelines for tags and stable branches
In upstream context we only run pipelines on staging branches, and
limited publishing jobs on the default branch.

We don't want to run pipelines on stable branches, or tags, because
the content will have already been tested on a staging branch before
getting pushed.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé e28112d007 gitlab: stable staging branches publish containers in a separate tag
If the stable staging branches publish containers under the 'latest' tag
they will clash with containers published on the primary staging branch,
as well  as with each other. This introduces logic that overrides the
container tag when jobs run against the stable staging branches.

The CI_COMMIT_REF_SLUG variable we use expands to the git branch name,
but with most special characters removed, such that it is valid as a
docker tag name. eg 'staging-8.0' will get a slug of 'staging-8-0'

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé 1ddd2ff9cd gitlab: allow overriding name of the upstream repository
The CI rules have special logic for what happens in upstream. To enable
contributors who modify CI rules to test this logic, however, they need
to be able to override which repo is considered upstream. This
introduces the 'QEMU_CI_UPSTREAM' variable

  git push gitlab <branch> -o ci.variable=QEMU_CI_UPSTREAM=berrange

to make it look as if my namespace is the actual upstream. Namespace in
this context refers to the path fragment in gitlab URLs that is above
the repository. Typically this will be the contributor's gitlab login
name.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-3-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé d4c7a56539 gitlab: centralize the container tag name
We use a fixed container tag of 'latest' so that contributors' forks
don't end up with an ever growing number of containers as they work
on throwaway feature branches.

This fixed tag causes problems running CI upstream in stable staging
branches, however, because the stable staging branch will publish old
container content that clashes with that needed by primary staging
branch. This makes it impossible to reliably run CI pipelines in
parallel in upstream for different staging branches.

This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to
change which tag container publishing uses. Initially it can be set
by contributors as a git push option if they want to override the
default use of 'latest' eg

  git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish

this is useful if contributors need to run pipelines for different
branches concurrently in their forks.

Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé 3c4fb17723 tests/qtest: make more migration pre-copy scenarios run non-live
There are 27 pre-copy live migration scenarios being tested. In all of
these we force non-convergence and run for one iteration, then let it
converge and wait for completion during the second (or following)
iterations. At 3 mbps bandwidth limit the first iteration takes a very
long time (~30 seconds).

While it is important to test the migration passes and convergence
logic, it is overkill to do this for all 27 pre-copy scenarios. The
TLS migration scenarios in particular are merely exercising different
code paths during connection establishment.

To optimize time taken, switch most of the test scenarios to run
non-live (ie guest CPUs paused) with no bandwidth limits. This gives
a massive speed up for most of the test scenarios.

For test coverage the following scenarios are unchanged

 * Precopy with UNIX sockets
 * Precopy with UNIX sockets and dirty ring tracking
 * Precopy with XBZRLE
 * Precopy with UNIX compress
 * Precopy with UNIX compress (nowait)
 * Precopy with multifd

On a test machine this reduces execution time from 13 minutes to
8 minutes.

Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-10-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé 95014994e1 tests/qtest: distinguish src/dst migration VM stop/resume events
The 'got_stop' and 'got_resume' global variables apply to the src and
dst migration VM respectively. Change their names to make this explicit
to developers.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-9-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé 266ea334b2 tests/qtest: capture RESUME events during migration
When running migration tests we monitor for a STOP event so we can skip
redundant waits. This will be needed for the RESUME event too shortly.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-8-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé aca0406958 tests/qtest: replace wait_command() with qtest_qmp_assert_success
Most usage of wait_command() is followed by qobject_unref(), which
is just a verbose re-implementation of qtest_qmp_assert_success().

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-7-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé cdf5ab5587 tests/qtest: switch to using event callbacks for STOP event
Change the migration test to use the new qtest event callback to watch
for the stop event. This ensures that we only watch for the STOP event
on the source QEMU. The previous code would set the single 'got_stop'
flag when either source or dest QEMU got the STOP event.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-6-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé 11936f0ef6 tests/qtest: get rid of some 'qtest_qmp' usage in migration test
Some of the usage is just a verbose way of re-inventing the
qtest_qmp_assert_success(_ref) methods.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-5-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé ffd4727589 tests/qtest: get rid of 'qmp_command' helper in migration test
This function duplicates logic of qtest_qmp_assert_success_ref.
The qtest_qmp_assert_success_ref method has better diagnostics
on failure because it prints the entire QMP response, instead
of just asserting on existance of the 'error' key.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-4-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé 0150e75d01 tests/qtest: add support for callback to receive QMP events
Currently code must call one of the qtest_qmp_event* functions to
fetch events. These are only usable if the immediate caller knows
the particular event they want to capture, and are only interested
in one specific event type. Adding ability to register an event
callback lets the caller capture a range of events over any period
of time.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-3-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé 28760edcd9 tests/qtest: add various qtest_qmp_assert_success() variants
Add several counterparts of qtest_qmp_assert_success() that can

 * Use va_list instead of ...
 * Accept a list of FDs to send
 * Return the response data

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230601161347.1803440-2-berrange@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-06-02 11:46:19 +02:00
Daniel P. Berrangé ab72522797 gitlab: switch from 'stable' to 'latest' docker container tags
The 'stable' and 'stable-dind' tags are not documented as supported
tags at:

  https://hub.docker.com/_/docker

Looking at their content they reflect docker 19.x.x release series,
were last built in Dec 2020, and have 3 critical and 20 high rated
CVEs unfixed. This obsolete status is attested by this commit:

  https://github.com/docker-library/docker/commit/606c63960a4845af7077721eb3900c706f5d0c5e

The 'stable-dind' tag in particular appears buggy as it is unable to
resolve DNS for Fedora repos:

  - Curl error (6): Couldn't resolve host name for https://mirrors.fedoraproject.org/metalink?repo=fedora-37&arch=x86_64&countme=1 [getaddrinfo() thread failed to start]

We used the 'stable' tag previously at the recommendation of GitLab
docs, but those docs are wrong and pending a fix:

  https://gitlab.com/gitlab-org/gitlab/-/issues/409430

Fixes: 5f63a67adb
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230531140654.1141145-1-berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-31 10:29:14 -07:00
Daniel P. Berrangé 855436dbf7 tests/qtest: replace qmp_discard_response with qtest_qmp_assert_success
The qmp_discard_response method simply ignores the result of the QMP
command, merely unref'ing the object. This is a bad idea for tests
as it leaves no trace if the QMP command unexpectedly failed. The
qtest_qmp_assert_success method will validate that the QMP command
returned without error, and if errors occur, it will print a message
on the console aiding debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230421171411.566300-2-berrange@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Zhang Chen <chen.zhang@intel.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-05-16 09:14:18 +02:00
Daniel P. Berrangé a17dbc4b79 qapi: allow unions to contain further unions
This extends the QAPI schema validation to permit unions inside unions,
provided the checks for clashing fields pass.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230420102619.348173-4-berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2023-04-26 07:52:45 +02:00
Daniel P. Berrangé 7ce54db230 qapi: support updating expected test output via make
It is possible to pass --update to tests/qapi-schema/test-qapi.py
to make it update the output files on error. This is inconvenient
to achieve though when test-qapi.py is run indirectly by make/meson.

Instead simply allow for an env variable to be set:

 $ QAPI_TEST_UPDATE= make check-qapi-schema

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230420102619.348173-2-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2023-04-25 15:23:06 +02:00
Daniel P. Berrangé bdd53274f2 tests/vm: use the default system python for NetBSD
Currently our NetBSD VM recipe requests instal of the python37 package
and explicitly tells QEMU to use that version of python. Since the
NetBSD base ISO was updated to version 9.3 though, the default system
python version is 3.9 which is sufficiently new for QEMU to rely on.
Rather than requesting an older python, just test against the default
system python which is what most users will have.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230329124601.822209-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230403134920.2132362-10-alex.bennee@linaro.org>
2023-04-04 15:56:44 +01:00
Daniel P. Berrangé 90834f5de6 tests/qemu-iotests: explicitly invoke 'check' via 'python'
The 'check' script will use "#!/usr/bin/env python3" by default
to locate python, but this doesn't work in distros which lack a
bare 'python3' binary like NetBSD.

We need to explicitly invoke 'check' by referring to the 'python'
variable in meson, which resolves to the detected python binary
that QEMU intends to use.

This fixes a regression introduced by

  commit 51ab5f8bd7
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Wed Mar 15 17:43:23 2023 +0000

    iotests: register each I/O test separately with meson

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230329124539.822022-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230403134920.2132362-9-alex.bennee@linaro.org>
2023-04-04 15:56:44 +01:00
Daniel P. Berrangé e35b9a2e81 qtests: avoid printing comments before g_test_init()
The TAP protocol version line must be the first thing printed on
stdout. The migration test failed that requirement in certain
scenarios:

  # Skipping test: Userfault not available (builtdtime)
  TAP version 13
  # random seed: R02Sc120c807f11053eb90bfea845ba1e368
  1..32
  # Start of x86_64 tests
  # Start of migration tests
  ....

The TAP version is printed by g_test_init(), so we need to make
sure that any methods which print are run after that.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20230317170553.592707-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 0d01a2f8a4 iotests: remove the check-block.sh script
Now that meson directly invokes the individual I/O tests, the
check-block.sh wrapper script is no longer required.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-9-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-26-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 51ab5f8bd7 iotests: register each I/O test separately with meson
Currently meson registers a single test that invokes an entire group of
I/O tests, hiding the test granularity from meson. There are various
downsides of doing this

 * You cannot ask 'meson test' to invoke a single I/O test
 * The meson test timeout can't be applied to the individual
   tests
 * Meson only gets a pass/fail for the overall I/O test group
   not individual tests
 * If a CI job gets killed by the GitLab timeout, we don't
   get visibility into how far through the I/O tests
   execution got.

This switches meson to perform test discovery by invoking 'check' in
dry-run mode. It then registers one meson test case for each I/O
test. Parallel execution remains disabled since the I/O tests do not
use self contained execution environments and thus conflict with
each other.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-8-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-25-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 5ba7db0938 iotests: always use a unique sub-directory per test
The current test runner is only safe against parallel execution within
a single instance of the 'check' process, and only if -j is given a
value greater than 2. This prevents running multiple copies of the
'check' process for different test scenarios.

This change switches the output / socket directories to always include
the test name, image format and image protocol. This should allow full
parallelism of all distinct test scenarios. eg running both qcow2 and
raw tests at the same time, or both file and nbd tests at the same
time.

It would be possible to allow for parallelism of the same test scenario
by including the pid, but that would potentially let many directories
accumulate over time on failures, so is not done.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-7-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-24-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé cb845eaa88 iotests: connect stdin to /dev/null when running tests
Currently the tests have their stdin inherited from the test harness,
meaning they are connected to a TTY. The QEMU processes spawned by
certain tests, however, modify TTY settings and if the test exits
abnormally the settings might not be restored.

The python test harness thus has some logic which will capture the
initial TTY settings and restore them once all tests are finished.

This does not, however, take into account the possibility of many
copies of the 'check' program running in parallel. With parallel
execution, a later invokation may save the TTY state that QEMU has
already modified, and thus restore bad state leaving the TTY
non-functional.

None of the I/O tests shnould actually be interactive requiring
user input and so they should not require a TTY at all. To avoid
this while TTY save/restore complexity we can connect the test
stdin to /dev/null instead.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-6-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-23-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 6e5792a1f6 iotests: print TAP protocol version when reporting tests
Recently meson started complaining that TAP test reports don't include
the TAP protocol version. While this warning is bogus and has since been
removed from Meson, it looks like good practice to include this header
going forward. The GLib library test harness has started unconditionally
printing the version, so this brings the I/O tests into line.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-5-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-22-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 663755b022 iotests: strip subdir path when listing tests
When asking 'check' to list individual tests by invoking it in dry run
mode, it prints the paths to the tests relative to the base of the
I/O test directory.

When asking 'check' to run an individual test, however, it mandates that
only the unqualified test name is given, without any path prefix. This
inconsistency makes it harder to ask for a list of tests and then invoke
each one.

Thus the test listing code is change to flatten the test names, by
printing only the base name, which can be directly invoked.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-4-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-21-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé a9e21786da iotests: allow test discovery before building
The 'check' script can be invoked in "dry run" mode, in which case it
merely does test discovery and prints out all their names. Despite only
doing test discovery it still validates that the various QEMU binaries
can be found. This makes it impossible todo test discovery prior to
building QEMU. This is a desirable feature to support, because it will
let meson discover tests.

Fortunately the code in the TestEnv constructor is ordered in a way
that makes this fairly trivial to achieve. We can just short circuit
the constructor after the basic directory paths have been set.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-20-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 0c8076b024 iotests: explicitly pass source/build dir to 'check' command
The 'check' script has some rather dubious logic whereby it assumes
that if invoked as a symlink, then it is running from a separate
source tree and build tree, otherwise it assumes the current working
directory is a combined source and build tree.

This doesn't work if you want to invoke the 'check' script using
its full source tree path while still using a split source and build
tree layout. This would be a typical situation with meson if you ask
it to find the 'check' script path using files('check').

Rather than trying to make the logic more magical, add support for
explicitly passing the dirs using --source-dir and --build-dir. If
either is omitted the current logic is maintained.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Hanna Czenczek <hreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230303160727.3977246-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-19-alex.bennee@linaro.org>
2023-03-22 15:08:26 +00:00
Daniel P. Berrangé 85b983485b tests/vm: custom openbsd partitioning to increase /home space
The openbsd image is 20GB in size, but the automatic partitioning
done by the installer leaves /home with a mere ~3.5 GB of space,
wasting free space across many other partitions that are not
used by our build process:

openbsd$ df
Filesystem  512-blocks      Used     Avail Capacity  Mounted on
/dev/sd0a      1229692    213592    954616    18%    /
/dev/sd0k      7672220        40   7288572     0%    /home
/dev/sd0d      1736604        24   1649752     0%    /tmp
/dev/sd0f      4847676   2505124   2100172    54%    /usr
/dev/sd0g      1326684    555656    704696    44%    /usr/X11R6
/dev/sd0h      4845436   1445932   3157236    31%    /usr/local
/dev/sd0j     10898972         4  10354020     0%    /usr/obj
/dev/sd0i      3343644         4   3176460     0%    /usr/src
/dev/sd0e      2601212     19840   2451312     1%    /var

This change tells the installer todo custom partitioning with
4 GB on /, 256 MB swap, and the remaining ~15GB for /home

openbsd$ df
Filesystem  512-blocks      Used     Avail Capacity  Mounted on
/dev/sd0a      7932412   4740204   2795588    63%    /
/dev/sd0d     32164636        40  30556368     0%    /home

This will avoid ENOSPC failures when tests that need to create
big files (disk images) run in parallel.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230322123639.836104-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2023-03-22 15:08:22 +00:00
Daniel P. Berrangé 3b67f43cf3 tests/vm: skip X11 in openbsd installation
As a VM used only for automated testing there is no need to
install the X11 stack.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230322123639.836104-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2023-03-22 15:08:18 +00:00
Daniel P. Berrangé 36debafddd ui: remove deprecated 'password' option for SPICE
This has been replaced by the 'password-secret' option,
which references a 'secret' object instance.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-02-15 11:14:58 -05:00
Daniel P. Berrangé 610783cb6e block: deprecate iSCSI 'password' in favour of 'password-secret'
Support for referencing secret objects was added in

  commit b189346eb1
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Thu Jan 21 14:19:21 2016 +0000

    iscsi: add support for getting CHAP password via QCryptoSecret API

The existing 'password' option is overdue for deprecation and
subsequent removal.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-02-15 11:01:04 -05:00
Daniel P. Berrangé c3b3a6c956 block: mention 'password-secret' option for -iscsi
The 'password-secret' option was added

  commit b189346eb1
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Thu Jan 21 14:19:21 2016 +0000

    iscsi: add support for getting CHAP password via QCryptoSecret API

but was not mentioned in the command line docs

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-02-15 11:01:04 -05:00
Daniel P. Berrangé 6003159ce1 Revert "linux-user: fix compat with glibc >= 2.36 sys/mount.h"
This reverts commit 3cd3df2a95.

glibc has fixed (in 2.36.9000-40-g774058d729) the problem
that caused a clash when both sys/mount.h annd linux/mount.h
are included, and backported this to the 2.36 stable release
too:

  https://sourceware.org/glibc/wiki/Release/2.36#Usage_of_.3Clinux.2Fmount.h.3E_and_.3Csys.2Fmount.h.3E

It is saner for QEMU to remove the workaround it applied for
glibc 2.36 and expect distros to ship the 2.36 maint release
with the fix. This avoids needing to add a further workaround
to QEMU to deal with the fact that linux/brtfs.h now also pulls
in linux/mount.h via linux/fs.h since Linux 6.1

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230110174901.2580297-3-berrange@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2023-02-03 22:55:12 +01:00
Daniel P. Berrangé 9f0246539a Revert "linux-user: add more compat ioctl definitions"
This reverts commit c5495f4ecb.

glibc has fixed (in 2.36.9000-40-g774058d729) the problem
that caused a clash when both sys/mount.h annd linux/mount.h
are included, and backported this to the 2.36 stable release
too:

  https://sourceware.org/glibc/wiki/Release/2.36#Usage_of_.3Clinux.2Fmount.h.3E_and_.3Csys.2Fmount.h.3E

It is saner for QEMU to remove the workaround it applied for
glibc 2.36 and expect distros to ship the 2.36 maint release
with the fix. This avoids needing to add a further workaround
to QEMU to deal with the fact that linux/brtfs.h now also pulls
in linux/mount.h via linux/fs.h since Linux 6.1

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230110174901.2580297-2-berrange@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2023-02-03 22:55:12 +01:00
Daniel P. Berrangé 7a92a8573c qapi, audio: Make introspection reflect build configuration more closely
Currently the -audiodev accepts any audiodev type regardless of what is
built in to QEMU. An error only occurs later at runtime when a sound
device tries to use the audio backend.

With this change QEMU will immediately reject -audiodev args that are
not compiled into the binary. The QMP schema will also be introspectable
to identify what is compiled in.

This also helps to avoid compiling code that is not required in the
binary. Note: When building the audiodevs as modules, the patch only
compiles out code for modules that we don't build at all.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[thuth: Rebase, take sndio and dbus devices into account]
Message-Id: <20230123083957.20349-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-01-30 15:43:55 +01:00
Daniel P. Berrangé 637d18090e qapi, audio: add query-audiodev command
Way back in QEMU 4.0, the -audiodev command line option was introduced
for configuring audio backends. This CLI option does not use QemuOpts
so it is not visible for introspection in 'query-command-line-options',
instead using the QAPI Audiodev type.  Unfortunately there is also no
QMP command that uses the Audiodev type, so it is not introspectable
with 'query-qmp-schema' either.

This introduces a 'query-audiodev' command that simply reflects back
the list of configured -audiodev command line options. This alone is
maybe not very useful by itself, but it makes Audiodev introspectable
via 'query-qmp-schema', so that libvirt (and other upper layer tools)
can discover the available audiodevs.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[thuth: Update for upcoming QEMU v8.0, and use QAPI_LIST_PREPEND]
Message-Id: <20230123083957.20349-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-01-30 15:43:48 +01:00
Daniel P. Berrangé 8b8437259c hw/display: avoid creating empty loadable modules
When using --disable-virglrenderer, QEMU still creates

  hw-display-virtio-gpu-gl.so
  hw-display-virtio-vga-gl.so
  hw-display-virtio-gpu-pci-gl.so

but when these are loaded, they provide no functionality as the code
which registers types is not compiled in. Funtionally this is
relatively harmless, because QEMU is fine loading a module with no
types.

This is rather confusing for users and OS distro maintainers though,
as they think they have the GL functionality built, but in fact the
module they are looking at provides nothing of value.

The root cause is the use of 'when/if_true' rules when adding sources
to the module source set. If all the rules evaluate to false, then we
have declared the module, but not added anything to it.  We need to
put declaration of the entire module inside a condition based on
existance of the 3rd party library deps that are mandatory.

Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1352
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221219125830.2369169-1-berrange@redhat.com>
[Do not check for pixman. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:34 +01:00
Daniel P. Berrangé 88a0ef00d7 enforce use of G_GNUC_PRINTF attributes
We've been very gradually adding G_GNUC_PRINTF annotations
to functions over years. This has been useful in detecting
certain malformed printf strings, or cases where we pass
user data as the printf format which is a potential security
flaw.

Given the inherant memory corruption danger in use of format
strings vs mis-matched variadic arguments, it is worth applying
G_GNUC_PRINTF to all functions using printf, even if we know
they are safe.

The compilers can reasonably reliably identify such places
with the -Wsuggest-attribute=format / -Wmissing-format-attribute
flags.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221219130205.687815-7-berrange@redhat.com>
[-Wsuggest-attribute=format and -Wmissing-format-attribute are
 synonyms, only include one; disable it for testfloat. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:34 +01:00
Daniel P. Berrangé 0472b2e541 tests: add G_GNUC_PRINTF for various functions
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221219130205.687815-6-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:34 +01:00
Daniel P. Berrangé beede7e848 util/error: add G_GNUC_PRINTF for various functions
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221219130205.687815-5-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:34 +01:00
Daniel P. Berrangé e4418354c0 tools/virtiofsd: add G_GNUC_PRINTF for logging functions
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221219130205.687815-4-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:34 +01:00
Daniel P. Berrangé d62449daf2 hw/xen: use G_GNUC_PRINTF/SCANF for various functions
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <20221219130205.687815-3-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:33 +01:00
Daniel P. Berrangé d94e96e7cf disas: add G_GNUC_PRINTF to gstring_printf
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221219130205.687815-2-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:33 +01:00
Daniel P. Berrangé 190973dc71 gitlab: remove redundant setting of PKG_CONFIG_PATH
The PKG_CONFIG_PATH variable is not defined in GitLab CI
envs and even if it was, we don't need to set it to its
existing value.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20221103173044.3969425-2-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-11 10:44:33 +01:00
Daniel P. Berrangé ee1c08bd73 ich9: honour 'enable_tco' property
An 'ICH9-LPC.enable_tco' property has been exposed for a
very long time, but attempts to set it have never been
honoured.

Originally, any user provided 'enable_tco' value was force
replaced by a value passed from the machine type setup
code that was determine by machine type compat properties.

  commit d6b304ba92
  Author: Eduardo Habkost <ehabkost@redhat.com>
  Date:   Sat Jan 23 14:02:10 2016 -0200

    machine: Remove no_tco field

    The field is always set to zero, so it is not necessary anymore.

After legacy Q35 machine types were deleted in:

  commit 86165b499e
  Author: Eduardo Habkost <ehabkost@redhat.com>
  Date:   Sat Jan 23 14:02:09 2016 -0200

    q35: Remove old machine versions

the machine type code ended up just unconditionally passing
'true', all the time, so this was further simplified in

  commit d6b304ba92
  Author: Eduardo Habkost <ehabkost@redhat.com>
  Date:   Sat Jan 23 14:02:10 2016 -0200

    machine: Remove no_tco field

    The field is always set to zero, so it is not necessary anymore.

  commit 18d6abae3e
  Author: Eduardo Habkost <ehabkost@redhat.com>
  Date:   Sat Jan 23 14:02:11 2016 -0200

    ich9: Remove enable_tco arguments from init functions

    The enable_tco arguments are always true, so they are not needed
    anymore.

Leaving the ich9_pm_init to just force set 'enable_tco' to true.
This still overrides any user specified property. The initialization
of property defaults should be done when properties are first
registered, rather than during object construction.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221216125749.596075-6-berrange@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21 06:35:28 -05:00
Daniel P. Berrangé a6b6414f0c hw/isa: enable TCO watchdog reboot pin strap by default
The TCO watchdog implementation default behaviour from POV of the
guest OS relies on the initial values for two I/O ports:

  * TCO1_CNT == 0x0

    Since bit 11 (TCO Timer Halt) is clear, the watchdog state
    is considered to be initially running

  * GCS == 0x20

    Since bit 5 (No Reboot) is set, the watchdog will not trigger
    when the timer expires

This is a safe default, because the No Reboot bit will prevent the
watchdog from triggering if the guest OS is unaware of its existance,
or is slow in configuring it. When a Linux guest initializes the TCO
watchdog, it will attempt to clear the "No Reboot" flag, and read the
value back. If the clear was honoured, the driver will treat this as
an indicator that the watchdog is functional and create the guest
watchdog device.

QEMU implements a second "no reboot" flag, however, via pin straps
which overrides the behaviour of the guest controlled "no reboot"
flag:

  commit 5add35bec1
  Author: Paulo Alcantara <pcacjr@gmail.com>
  Date:   Sun Jun 28 14:58:58 2015 -0300

    ich9: implement strap SPKR pin logic

This second 'noreboot' pin was defaulted to high, which also inhibits
triggering of the requested watchdog actions, unless QEMU is launched
with the magic flag "-global ICH9-LPC.noreboot=false".

This is a bad default as we are exposing a watchdog to every guest OS
using the q35 machine type, but preventing it from actually doing what
it is designed to do. What is worse is that the guest OS and its apps
have no way to know that the watchdog is never going to fire, due to
this second 'noreboot' pin.

If a guest OS had no watchdog device at all, then apps whose operation
and/or data integrity relies on a watchdog can refuse to launch, and
alert the administrator of the problematic deployment. With Q35 machines
unconditionally exposing a watchdog though, apps will think their
deployment is correct but in fact have no protection at all.

This patch flips the default of the second 'no reboot' flag, so that
configured watchdog actions will be honoured out of the box for the
7.2 Q35 machine type onwards, if the guest enables use of the watchdog.

See also related bug reports

  https://bugzilla.redhat.com/show_bug.cgi?id=2080207
  https://bugzilla.redhat.com/show_bug.cgi?id=2136889
  https://bugzilla.redhat.com/show_bug.cgi?id=2137346

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221216125749.596075-5-berrange@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21 06:35:28 -05:00
Daniel P. Berrangé 6f10a29e76 hw/watchdog: add trace events for watchdog action handling
The tracepoints aid in debugging the triggering of watchdog devices.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221216125749.596075-4-berrange@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21 06:35:28 -05:00
Daniel P. Berrangé c8c7c406db hw/isa: add trace events for ICH9 LPC chip config access
These tracepoints aid in understanding and debugging the guest drivers
for the TCO watchdog.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221216125749.596075-3-berrange@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21 06:35:28 -05:00
Daniel P. Berrangé 23d8e32499 hw/acpi: add trace events for TCO watchdog register access
These tracepoints aid in understanding and debugging the guest drivers
for the TCO watchdog.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221216125749.596075-2-berrange@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21 06:35:28 -05:00
Daniel P. Berrangé c70fe3b148 ci: replace x86_64 macos-11 with aarch64 macos-12
The Cirrus CI service has announced the intent to discontinue
support for x86_64 macOS CI runners. They already have aarch64
runners available and require all projects to switch to these
images before Jan 1st 2023. The different architecture is
merely determined by the image name requested.

For aarch64 they only support macOS 12 onwards. At the same
time our support policy only guarantees the most recent 2
major versions, so macOS 12 is already technically our min
version.

https://cirrus-ci.org/blog/2022/11/08/sunsetting-intel-macos-instances/

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221116175023.80627-1-berrange@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-17 09:58:11 +01:00
Daniel P. Berrangé 7d0e12af59 net: improve error message for missing netdev backend
The current message when using '-net user...' with SLIRP disabled at
compile time is:

  qemu-system-x86_64: -net user: Parameter 'type' expects a net backend type (maybe it is not compiled into this binary)

An observation is that we're using the 'netdev->type' field here which
is an enum value, produced after QAPI has converted from its string
form.

IOW, at this point in the code, we know that the user's specified
type name was a valid network backend. The only possible scenario that
can make the backend init function be NULL, is if support for that
backend was disabled at build time. Given this, we don't need to caveat
our error message with a 'maybe' hint, we can be totally explicit.

The use of QERR_INVALID_PARAMETER_VALUE doesn't really lend itself to
user friendly error message text. Since this is not used to set a
specific QAPI error class, we can simply stop using this pre-formatted
error text and provide something better.

Thus the new message is:

  qemu-system-x86_64: -net user: network backend 'user' is not compiled into this binary

The case of passing 'hubport' for -net is also given a message reminding
people they should have used -netdev/-nic instead, as this backend type
is only valid for the modern syntax.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-10-28 13:28:52 +08:00
Daniel P. Berrangé da0ab2c4c4 crypto: add test cases for many malformed LUKS header scenarios
Validate that we diagnose each malformed LUKS header scenario with a
distinct error report.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 13:06:12 +01:00
Daniel P. Berrangé 741c314a33 crypto: ensure LUKS tests run with GNUTLS crypto provider
GNUTLS is supported as a crypto provider since

  commit cc4c7c7382
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Wed Jun 30 17:20:02 2021 +0100

    crypto: introduce build system for gnutls crypto backend

So enable the LUKS tests in this config.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 13:06:11 +01:00
Daniel P. Berrangé 6c1989321e crypto: quote algorithm names in error messages
If given a malformed LUKS header, it is possible that the algorithm
names end up being an empty string. This leads to confusing error
messages unless quoting is used to highlight where the empty string
is subsituted in the error message.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé 98c72dfb71 crypto: split off helpers for converting LUKS header endianess
The unit test suite is shortly going to want to convert header
endianness separately from the main I/O functions.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé 36445acebd crypto: split LUKS header definitions off into file
This will allow unit testing code to use the structs.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé b57151ac03 crypto: check that LUKS PBKDF2 iterations count is non-zero
Both the master key and key slot passphrases are run through the PBKDF2
algorithm. The iterations count is expected to be generally very large
(many 10's or 100's of 1000s). It is hard to define a low level cutoff,
but we can certainly say that iterations count should be non-zero. A
zero count likely indicates an initialization mistake so reject it.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé c5f6962801 crypto: strengthen the check for key slots overlapping with LUKS header
The LUKS header data on disk is a fixed size, however, there's expected
to be a gap between the end of the header and the first key slot to get
alignment with the 2nd sector on 4k drives. This wasn't originally part
of the LUKS spec, but was always part of the reference implementation,
so it is worth validating this.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé d233fbc327 crypto: validate that LUKS payload doesn't overlap with header
We already validate that LUKS keyslots don't overlap with the
header, or with each other. This closes the remaining hole in
validation of LUKS file regions.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé 93569c3730 crypto: enforce that key material doesn't overlap with LUKS header
We already check that key material doesn't overlap between key slots,
and that it doesn't overlap with the payload. We didn't check for
overlap with the LUKS header.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé f1195961f3 crypto: enforce that LUKS stripes is always a fixed value
Although the LUKS stripes are encoded in the keyslot header and so
potentially configurable, in pratice the cryptsetup impl mandates
this has the fixed value 4000. To avoid incompatibility apply the
same enforcement in QEMU too. This also caps the memory usage for
key material when QEMU tries to open a LUKS volume.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé c1d8634c20 crypto: sanity check that LUKS header strings are NUL-terminated
The LUKS spec requires that header strings are NUL-terminated, and our
code relies on that. Protect against maliciously crafted headers by
adding validation.

Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 12:55:27 +01:00
Daniel P. Berrangé f1018ea0a3 tests: avoid DOS line endings in PSK file
Using FILE * APIs for writing the PSK file results in translation from
UNIX to DOS line endings on Windows. When the crypto PSK code later
loads the credentials the stray \r will result in failure to load the
PSK credentials into GNUTLS.

Rather than switching the FILE* APIs to open in binary format, just
switch to the more concise g_file_set_contents API.

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 11:55:41 +01:00
Daniel P. Berrangé 3983bf1b41 crypto: check for and report errors setting PSK credentials
If setting credentials fails, the handshake will later fail to complete
with an obscure error message which is hard to diagnose.

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 11:55:41 +01:00
Daniel P. Berrangé dd84a906e0 scripts: check if .git exists before checking submodule status
Currently we check status of each submodule, before actually checking
if we're in a git repo. These status commands will all fail, but we
are hiding their output so we don't see it currently.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-10-27 11:54:37 +01:00
Daniel P. Berrangé ed98cdecf8 linux-user: remove conditionals for many fs.h ioctls
These ioctls have been defined in linux/fs.h for a long time

  * BLKGETSIZE64 - <2.6.12 (linux.git epoch)
  * BLKDISCARD - 2.6.28 (d30a2605be9d5132d95944916e8f578fcfe4f976)
  * BLKIOMIN - 2.6.32 (ac481c20ef8f6c6f2be75d581863f40c43874ef7)
  * BLKIOOPT - 2.6.32 (ac481c20ef8f6c6f2be75d581863f40c43874ef7)
  * BLKALIGNOFF - 2.6.32 (ac481c20ef8f6c6f2be75d581863f40c43874ef7)
  * BLKPBSZGET - 2.6.32 (ac481c20ef8f6c6f2be75d581863f40c43874ef7)
  * BLKDISCARDZEROES - 2.6.32 (98262f2762f0067375f83824d81ea929e37e6bfe)
  * BLKSECDISCARD - 2.6.36 (8d57a98ccd0b4489003473979da8f5a1363ba7a3)
  * BLKROTATIONAL - 3.2 (ef00f59c95fe6e002e7c6e3663cdea65e253f4cc)
  * BLKZEROOUT - 3.6 (66ba32dc167202c3cf8c86806581a9393ec7f488)
  * FIBMAP - <2.6.12 (linux.git epoch)
  * FIGETBSZ - <2.6.12 (linux.git epoch)

and when building with latest glibc, we'll see compat definitions
in syscall.c anyway thanks to the previous patch. Thus we can
assume they always exist and remove the conditional checks.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20221004093206.652431-3-berrange@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-10-21 17:46:19 +02:00
Daniel P. Berrangé c5495f4ecb linux-user: add more compat ioctl definitions
GLibc changes prevent us from including linux/fs.h anymore,
and we previously adjusted to this in

  commit 3cd3df2a95
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Tue Aug 2 12:41:34 2022 -0400

    linux-user: fix compat with glibc >= 2.36 sys/mount.h

That change required adding compat ioctl definitions on the
QEMU side for any ioctls that we would otherwise obtain
from linux/fs.h.  This commit adds more that were initially
missed, due to their usage being conditionalized in QEMU.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20221004093206.652431-2-berrange@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-10-21 17:46:19 +02:00
Daniel P. Berrangé d135f78140 linux-user: use 'max' instead of 'qemu32' / 'qemu64' by default
The 'qemu64' CPU model implements the least featureful x86_64 CPU that's
possible. Historically this hasn't been an issue since it was rare for
OS distros to build with a higher mandatory CPU baseline.

With RHEL-9, however, the entire distro is built for the x86_64-v2 ABI
baseline:

  https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level

It is likely that other distros may take similar steps in the not too
distant future. For example, it has been suggested for Fedora on a
number of occasions.

This new baseline is not compatible with the qemu64 CPU model though.
While it is possible to pass a '-cpu xxx' flag to qemu-x86_64, the
usage of QEMU doesn't always allow for this. For example, the args
are typically controlled via binfmt rules that the user has no ability
to change. This impacts users who are trying to use podman on aarch64
platforms, to run containers with x86_64 content. There's no arg to
podman that can be used to change the qemu-x86_64 args, and a non-root
user of podman can not change binfmt rules without elevating privileges:

  https://github.com/containers/podman/issues/15456#issuecomment-1228210973

Changing to the 'max' CPU model gives 'qemu-x86_64' maximum
compatibility with binaries it is likely to encounter in the wild,
and not likely to have a significant downside for existing usage.

Most other architectures already use an 'any' CPU model, which is
often mapped to 'max' (or similar) already, rather than the oldest
possible CPU model.

For the sake of consistency the 'i386' architecture is also changed
from using 'qemu32' to 'max'.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220923110413.70593-1-berrange@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-09-23 14:04:17 +02:00
Daniel P. Berrangé 9c23d71934 tests/qtest: misc tweaks to readconfig
The property name parameter is ignored when visiting a top
level type, but the obvious typo should be fixed to avoid
confusion. A few indentation issues were tidied up. We
can break out of the loop when finding the RNG device.
Finally, close the temp FD immediately when no longer
needed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220809093854.168438-1-berrange@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-08-16 11:36:21 +02:00
Daniel P. Berrangé 3cd3df2a95 linux-user: fix compat with glibc >= 2.36 sys/mount.h
The latest glibc 2.36 has extended sys/mount.h so that it
defines the FSCONFIG_* enum constants. These are historically
defined in linux/mount.h, and thus if you include both headers
the compiler complains:

In file included from /usr/include/linux/fs.h:19,
                 from ../linux-user/syscall.c:98:
/usr/include/linux/mount.h:95:6: error: redeclaration of 'enum fsconfig_command'
   95 | enum fsconfig_command {
      |      ^~~~~~~~~~~~~~~~
In file included from ../linux-user/syscall.c:31:
/usr/include/sys/mount.h:189:6: note: originally defined here
  189 | enum fsconfig_command
      |      ^~~~~~~~~~~~~~~~
/usr/include/linux/mount.h:96:9: error: redeclaration of enumerator 'FSCONFIG_SET_FLAG'
   96 |         FSCONFIG_SET_FLAG       = 0,    /* Set parameter, supplying no value */
      |         ^~~~~~~~~~~~~~~~~
/usr/include/sys/mount.h:191:3: note: previous definition of 'FSCONFIG_SET_FLAG' with type 'enum fsconfig_command'
  191 |   FSCONFIG_SET_FLAG       = 0,    /* Set parameter, supplying no value */
      |   ^~~~~~~~~~~~~~~~~
...snip...

QEMU doesn't include linux/mount.h, but it does use
linux/fs.h and thus gets linux/mount.h indirectly.

glibc acknowledges this problem but does not appear to
be intending to fix it in the forseeable future, simply
documenting it as a known incompatibility with no
workaround:

  https://sourceware.org/glibc/wiki/Release/2.36#Usage_of_.3Clinux.2Fmount.h.3E_and_.3Csys.2Fmount.h.3E
  https://sourceware.org/glibc/wiki/Synchronizing_Headers

To address this requires either removing use of sys/mount.h
or linux/fs.h, despite QEMU needing declarations from
both.

This patch removes linux/fs.h, meaning we have to define
various FS_IOC constants that are now unavailable.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20220802164134.1851910-1-berrange@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-08-10 18:37:46 +02:00
Daniel P. Berrangé f6a5f38062 tests/qtest: add scenario for -readconfig handling
This test of -readconfig validates the last three regressions we
have fixed with -readconfig:

 * Interpretation of memory size units as MiB not bytes
 * Allow use of [spice]
 * Allow use of [object]

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220805115529.124544-2-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-08 14:54:56 +02:00
Daniel P. Berrangé 977c33ba5d target/arm: display deprecation status in '-cpu help'
When the user queries CPU models via QMP there is a 'deprecated' flag
present, however, this is not done for the CLI '-cpu help' command.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-08-05 16:18:15 +01:00
Daniel P. Berrangé 738cdc2f6e target/s390x: display deprecation status in '-cpu help'
When the user queries CPU models via QMP there is a 'deprecated' flag
present, however, this is not done for the CLI '-cpu help' command.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-08-05 16:18:15 +01:00