This allows plugins to specify their binary dependencies in bindep
format.
Some thinking on the implementation: this is in contrast to the
files/[deb|rpm] installation, which is called from the external
install_prereqs.sh script. This script being an externally callable
entry-point is really an artifact of the days when we would build
snapshot images for CI and wanted to pre-cache downloads. These days
we use the mirror system to keep packages close to CI nodes. Thus
rather than expand install_prereqs.sh to also be installing
virtualenvs and python dependencies, this seems to fit better as a
separate internal phase of stack.sh.
Documentation is updated
Change-Id: Icbdfbf97c17c906a7ae86f43e80eb2c445816228
| ... | ... |
@@ -222,14 +222,20 @@ dependency mechanism is beyond the scope of the current work. |
| 222 | 222 |
System Packages |
| 223 | 223 |
=============== |
| 224 | 224 |
|
| 225 |
-Devstack provides a framework for getting packages installed at an early |
|
| 226 |
-phase of its execution. These packages may be defined in a plugin as files |
|
| 227 |
-that contain new-line separated lists of packages required by the plugin |
|
| 228 | 225 |
|
| 229 |
-Supported packaging systems include apt and yum across multiple distributions. |
|
| 230 |
-To enable a plugin to hook into this and install package dependencies, packages |
|
| 231 |
-may be listed at the following locations in the top-level of the plugin |
|
| 232 |
-repository: |
|
| 226 |
+ |
|
| 227 |
+Devstack based |
|
| 228 |
+-------------- |
|
| 229 |
+ |
|
| 230 |
+Devstack provides a custom framework for getting packages installed at |
|
| 231 |
+an early phase of its execution. These packages may be defined in a |
|
| 232 |
+plugin as files that contain new-line separated lists of packages |
|
| 233 |
+required by the plugin |
|
| 234 |
+ |
|
| 235 |
+Supported packaging systems include apt and yum across multiple |
|
| 236 |
+distributions. To enable a plugin to hook into this and install |
|
| 237 |
+package dependencies, packages may be listed at the following |
|
| 238 |
+locations in the top-level of the plugin repository: |
|
| 233 | 239 |
|
| 234 | 240 |
- ``./devstack/files/debs/$plugin_name`` - Packages to install when running |
| 235 | 241 |
on Ubuntu, Debian or Linux Mint. |
| ... | ... |
@@ -240,6 +246,42 @@ repository: |
| 240 | 240 |
- ``./devstack/files/rpms-suse/$plugin_name`` - Packages to install when |
| 241 | 241 |
running on SUSE Linux or openSUSE. |
| 242 | 242 |
|
| 243 |
+Although there a no plans to remove this method of installing |
|
| 244 |
+packages, plugins should consider it deprecated for ``bindep`` support |
|
| 245 |
+described below. |
|
| 246 |
+ |
|
| 247 |
+bindep |
|
| 248 |
+------ |
|
| 249 |
+ |
|
| 250 |
+The `bindep <https://docs.openstack.org/infra/bindep>`__ project has |
|
| 251 |
+become the defacto standard for OpenStack projects to specify binary |
|
| 252 |
+dependencies. |
|
| 253 |
+ |
|
| 254 |
+A plugin may provide a ``./devstack/files/bindep.txt`` file, which |
|
| 255 |
+will be called with the *default* profile to install packages. For |
|
| 256 |
+details on the syntax, etc. see the bindep documentation. |
|
| 257 |
+ |
|
| 258 |
+It is also possible to use the ``bindep.txt`` of projects that are |
|
| 259 |
+being installed from source with the ``-bindep`` flag available in |
|
| 260 |
+install functions. For example |
|
| 261 |
+ |
|
| 262 |
+.. code-block:: bash |
|
| 263 |
+ |
|
| 264 |
+ if use_library_from_git "diskimage-builder"; then |
|
| 265 |
+ GITREPO["diskimage-builder"]=$DISKIMAGE_BUILDER_REPO_URL |
|
| 266 |
+ GITDIR["diskimage-builder"]=$DEST/diskimage-builder |
|
| 267 |
+ GITBRANCH["diskimage-builder"]=$DISKIMAGE_BUILDER_REPO_REF |
|
| 268 |
+ git_clone_by_name "diskimage-builder" |
|
| 269 |
+ setup_dev_lib -bindep "diskimage-builder" |
|
| 270 |
+ fi |
|
| 271 |
+ |
|
| 272 |
+will result in any packages required by the ``bindep.txt`` of the |
|
| 273 |
+``diskimage-builder`` project being installed. Note however that jobs |
|
| 274 |
+that switch projects between source and released/pypi installs |
|
| 275 |
+(e.g. with a ``foo-dsvm`` and a ``foo-dsvm-src`` test to cover both |
|
| 276 |
+released dependencies and master versions) will have to deal with |
|
| 277 |
+``bindep.txt`` being unavailable without the source directory. |
|
| 278 |
+ |
|
| 243 | 279 |
|
| 244 | 280 |
Using Plugins in the OpenStack Gate |
| 245 | 281 |
=================================== |
| ... | ... |
@@ -1248,6 +1248,30 @@ function get_plugin_packages {
|
| 1248 | 1248 |
$xtrace |
| 1249 | 1249 |
} |
| 1250 | 1250 |
|
| 1251 |
+# Search plugins for a bindep.txt file |
|
| 1252 |
+# |
|
| 1253 |
+# Uses globals ``BINDEP_CMD``, ``GITDIR``, ``DEVSTACK_PLUGINS`` |
|
| 1254 |
+# |
|
| 1255 |
+# Note this is only valid after BINDEP_CMD is setup in stack.sh, and |
|
| 1256 |
+# is thus not really intended to be called externally. |
|
| 1257 |
+function _get_plugin_bindep_packages {
|
|
| 1258 |
+ local xtrace |
|
| 1259 |
+ xtrace=$(set +o | grep xtrace) |
|
| 1260 |
+ set +o xtrace |
|
| 1261 |
+ |
|
| 1262 |
+ local bindep_file |
|
| 1263 |
+ local packages |
|
| 1264 |
+ |
|
| 1265 |
+ for plugin in ${DEVSTACK_PLUGINS//,/ }; do
|
|
| 1266 |
+ bindep_file=${GITDIR[$plugin]}/devstack/files/bindep.txt
|
|
| 1267 |
+ if [[ -f ${bindep_file} ]]; then
|
|
| 1268 |
+ packages+=$($BINDEP_CMD -b --file ${bindep_file} || true)
|
|
| 1269 |
+ fi |
|
| 1270 |
+ done |
|
| 1271 |
+ echo "${packages}"
|
|
| 1272 |
+ $xtrace |
|
| 1273 |
+} |
|
| 1274 |
+ |
|
| 1251 | 1275 |
# Distro-agnostic package installer |
| 1252 | 1276 |
# Uses globals ``NO_UPDATE_REPOS``, ``REPOS_UPDATED``, ``RETRY_UPDATE`` |
| 1253 | 1277 |
# install_package package [package ...] |
| ... | ... |
@@ -805,6 +805,13 @@ install_infra |
| 805 | 805 |
$VIRTUALENV_CMD $DEST/bindep-venv |
| 806 | 806 |
# TODO(ianw) : optionally install from zuul checkout? |
| 807 | 807 |
$DEST/bindep-venv/bin/pip install bindep |
| 808 |
+export BINDEP_CMD=${DEST}/bindep-venv/bin/bindep
|
|
| 809 |
+ |
|
| 810 |
+# Install packages as defined in plugin bindep.txt files |
|
| 811 |
+pkgs="$( _get_plugin_bindep_packages )" |
|
| 812 |
+if [[ -n "${pkgs}" ]]; then
|
|
| 813 |
+ install_package ${pkgs}
|
|
| 814 |
+fi |
|
| 808 | 815 |
|
| 809 | 816 |
# Extras Pre-install |
| 810 | 817 |
# ------------------ |