Browse code

Wait for compute service to check in

With cell v2, on initial bring up, discover hosts can't run unless all
the compute nodes have checked in. The documentation says that you
should run ``nova service-list --binary nova-compute`` and see all
your hosts before running discover hosts. This isn't really viable in
a multinode devstack because of how things are brought up in parts.

We can however know that stack.sh will not complete before the compute
node is up by waiting for the compute node to check in before moving
forward. This puts a few more seconds into the run, but ensures
everything is solid in multinode environments.

Change-Id: I667e6a9be3fee8bb5bfd73426eef567489e3d88d

Sean Dague authored on 2017/07/26 08:51:08
Showing 2 changed files
... ...
@@ -407,6 +407,20 @@ EOF
407 407
     return $rval
408 408
 }
409 409
 
410
+function wait_for_compute {
411
+    local timeout=$1
412
+    time_start "wait_for_service"
413
+    timeout $timeout bash -x <<EOF || rval=$?
414
+        ID=""
415
+        while [[ "\$ID" == "" ]]; do
416
+            sleep 1
417
+            ID=\$(openstack --os-cloud devstack-admin --os-region "$REGION_NAME" compute service list --host `hostname` --service nova-compute -c ID -f value)
418
+        done
419
+EOF
420
+    time_stop "wait_for_service"
421
+    return $rval
422
+}
423
+
410 424
 
411 425
 # ping check
412 426
 # Uses globals ``ENABLED_SERVICES``, ``TOP_DIR``, ``MULTI_HOST``, ``PRIVATE_NETWORK``
... ...
@@ -795,6 +795,13 @@ function start_nova_compute {
795 795
         fi
796 796
         run_process n-cpu "$NOVA_BIN_DIR/nova-compute --config-file $compute_cell_conf"
797 797
     fi
798
+    # NOTE(sdague): with cells v2 all the compute services must be up
799
+    # and checked into the database before discover_hosts is run. This
800
+    # happens in all in one installs by accident, because > 30 seconds
801
+    # happen between here and the script ending. However, in multinode
802
+    # tests this can very often not be the case. So ensure that the
803
+    # compute is up before we move on.
804
+    wait_for_compute 60
798 805
 
799 806
     export PATH=$old_path
800 807
 }