This allows plugins to specify their binary dependencies in bindep
format.
Some thinking on the implementation: this is in contrast to the
files/[deb|rpm] installation, which is called from the external
install_prereqs.sh script. This script being an externally callable
entry-point is really an artifact of the days when we would build
snapshot images for CI and wanted to pre-cache downloads. These days
we use the mirror system to keep packages close to CI nodes. Thus
rather than expand install_prereqs.sh to also be installing
virtualenvs and python dependencies, this seems to fit better as a
separate internal phase of stack.sh.
Documentation is updated
Change-Id: Icbdfbf97c17c906a7ae86f43e80eb2c445816228
... | ... |
@@ -222,14 +222,20 @@ dependency mechanism is beyond the scope of the current work. |
222 | 222 |
System Packages |
223 | 223 |
=============== |
224 | 224 |
|
225 |
-Devstack provides a framework for getting packages installed at an early |
|
226 |
-phase of its execution. These packages may be defined in a plugin as files |
|
227 |
-that contain new-line separated lists of packages required by the plugin |
|
228 | 225 |
|
229 |
-Supported packaging systems include apt and yum across multiple distributions. |
|
230 |
-To enable a plugin to hook into this and install package dependencies, packages |
|
231 |
-may be listed at the following locations in the top-level of the plugin |
|
232 |
-repository: |
|
226 |
+ |
|
227 |
+Devstack based |
|
228 |
+-------------- |
|
229 |
+ |
|
230 |
+Devstack provides a custom framework for getting packages installed at |
|
231 |
+an early phase of its execution. These packages may be defined in a |
|
232 |
+plugin as files that contain new-line separated lists of packages |
|
233 |
+required by the plugin |
|
234 |
+ |
|
235 |
+Supported packaging systems include apt and yum across multiple |
|
236 |
+distributions. To enable a plugin to hook into this and install |
|
237 |
+package dependencies, packages may be listed at the following |
|
238 |
+locations in the top-level of the plugin repository: |
|
233 | 239 |
|
234 | 240 |
- ``./devstack/files/debs/$plugin_name`` - Packages to install when running |
235 | 241 |
on Ubuntu, Debian or Linux Mint. |
... | ... |
@@ -240,6 +246,42 @@ repository: |
240 | 240 |
- ``./devstack/files/rpms-suse/$plugin_name`` - Packages to install when |
241 | 241 |
running on SUSE Linux or openSUSE. |
242 | 242 |
|
243 |
+Although there a no plans to remove this method of installing |
|
244 |
+packages, plugins should consider it deprecated for ``bindep`` support |
|
245 |
+described below. |
|
246 |
+ |
|
247 |
+bindep |
|
248 |
+------ |
|
249 |
+ |
|
250 |
+The `bindep <https://docs.openstack.org/infra/bindep>`__ project has |
|
251 |
+become the defacto standard for OpenStack projects to specify binary |
|
252 |
+dependencies. |
|
253 |
+ |
|
254 |
+A plugin may provide a ``./devstack/files/bindep.txt`` file, which |
|
255 |
+will be called with the *default* profile to install packages. For |
|
256 |
+details on the syntax, etc. see the bindep documentation. |
|
257 |
+ |
|
258 |
+It is also possible to use the ``bindep.txt`` of projects that are |
|
259 |
+being installed from source with the ``-bindep`` flag available in |
|
260 |
+install functions. For example |
|
261 |
+ |
|
262 |
+.. code-block:: bash |
|
263 |
+ |
|
264 |
+ if use_library_from_git "diskimage-builder"; then |
|
265 |
+ GITREPO["diskimage-builder"]=$DISKIMAGE_BUILDER_REPO_URL |
|
266 |
+ GITDIR["diskimage-builder"]=$DEST/diskimage-builder |
|
267 |
+ GITBRANCH["diskimage-builder"]=$DISKIMAGE_BUILDER_REPO_REF |
|
268 |
+ git_clone_by_name "diskimage-builder" |
|
269 |
+ setup_dev_lib -bindep "diskimage-builder" |
|
270 |
+ fi |
|
271 |
+ |
|
272 |
+will result in any packages required by the ``bindep.txt`` of the |
|
273 |
+``diskimage-builder`` project being installed. Note however that jobs |
|
274 |
+that switch projects between source and released/pypi installs |
|
275 |
+(e.g. with a ``foo-dsvm`` and a ``foo-dsvm-src`` test to cover both |
|
276 |
+released dependencies and master versions) will have to deal with |
|
277 |
+``bindep.txt`` being unavailable without the source directory. |
|
278 |
+ |
|
243 | 279 |
|
244 | 280 |
Using Plugins in the OpenStack Gate |
245 | 281 |
=================================== |
... | ... |
@@ -1248,6 +1248,30 @@ function get_plugin_packages { |
1248 | 1248 |
$xtrace |
1249 | 1249 |
} |
1250 | 1250 |
|
1251 |
+# Search plugins for a bindep.txt file |
|
1252 |
+# |
|
1253 |
+# Uses globals ``BINDEP_CMD``, ``GITDIR``, ``DEVSTACK_PLUGINS`` |
|
1254 |
+# |
|
1255 |
+# Note this is only valid after BINDEP_CMD is setup in stack.sh, and |
|
1256 |
+# is thus not really intended to be called externally. |
|
1257 |
+function _get_plugin_bindep_packages { |
|
1258 |
+ local xtrace |
|
1259 |
+ xtrace=$(set +o | grep xtrace) |
|
1260 |
+ set +o xtrace |
|
1261 |
+ |
|
1262 |
+ local bindep_file |
|
1263 |
+ local packages |
|
1264 |
+ |
|
1265 |
+ for plugin in ${DEVSTACK_PLUGINS//,/ }; do |
|
1266 |
+ bindep_file=${GITDIR[$plugin]}/devstack/files/bindep.txt |
|
1267 |
+ if [[ -f ${bindep_file} ]]; then |
|
1268 |
+ packages+=$($BINDEP_CMD -b --file ${bindep_file} || true) |
|
1269 |
+ fi |
|
1270 |
+ done |
|
1271 |
+ echo "${packages}" |
|
1272 |
+ $xtrace |
|
1273 |
+} |
|
1274 |
+ |
|
1251 | 1275 |
# Distro-agnostic package installer |
1252 | 1276 |
# Uses globals ``NO_UPDATE_REPOS``, ``REPOS_UPDATED``, ``RETRY_UPDATE`` |
1253 | 1277 |
# install_package package [package ...] |
... | ... |
@@ -805,6 +805,13 @@ install_infra |
805 | 805 |
$VIRTUALENV_CMD $DEST/bindep-venv |
806 | 806 |
# TODO(ianw) : optionally install from zuul checkout? |
807 | 807 |
$DEST/bindep-venv/bin/pip install bindep |
808 |
+export BINDEP_CMD=${DEST}/bindep-venv/bin/bindep |
|
809 |
+ |
|
810 |
+# Install packages as defined in plugin bindep.txt files |
|
811 |
+pkgs="$( _get_plugin_bindep_packages )" |
|
812 |
+if [[ -n "${pkgs}" ]]; then |
|
813 |
+ install_package ${pkgs} |
|
814 |
+fi |
|
808 | 815 |
|
809 | 816 |
# Extras Pre-install |
810 | 817 |
# ------------------ |