Browse code

Change the restart_rpc_backend loop to accomodate async rabbitmq

Some distros have converted to systemd for starting RabbitMQ. This has
resulted in:
---
[Call Trace]
./stack.sh:904:restart_rpc_backend
/home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die
[ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password
Error on exit
World dumping... see /opt/stack/logs/worlddump-2015-05-29-031618.txt for details
---

Because 'restart_service rabbitmq-server' returns before the server is ready to
accept connections.

Alter the retry loop to only restart the rabbitmq-server every second time
through the loop. Allowing time for the slow rabbit to start.

Closes-Bug: 1449056
Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce

Tony Breeds authored on 2015/05/15 11:51:43
Showing 1 changed files
... ...
@@ -194,13 +194,22 @@ function restart_rpc_backend {
194 194
         # NOTE(bnemec): Retry initial rabbitmq configuration to deal with
195 195
         # the fact that sometimes it fails to start properly.
196 196
         # Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144100
197
+        # NOTE(tonyb): Extend the orginal retry logic to only restart rabbitmq
198
+        # every second time around the loop.
199
+        # See: https://bugs.launchpad.net/devstack/+bug/1449056 for details on
200
+        # why this is needed.  This can bee seen on vivid and Debian unstable
201
+        # (May 2015)
202
+        # TODO(tonyb): Remove this when Debian and Ubuntu have a fixed systemd
203
+        # service file.
197 204
         local i
198
-        for i in `seq 10`; do
205
+        for i in `seq 20`; do
199 206
             local rc=0
200 207
 
201
-            [[ $i -eq "10" ]] && die $LINENO "Failed to set rabbitmq password"
208
+            [[ $i -eq "20" ]] && die $LINENO "Failed to set rabbitmq password"
202 209
 
203
-            restart_service rabbitmq-server
210
+            if [[ $(( i % 2 )) == "0" ]] ; then
211
+                restart_service rabbitmq-server
212
+            fi
204 213
 
205 214
             rabbit_setuser "$RABBIT_USERID" "$RABBIT_PASSWORD" || rc=$?
206 215
             if [ $rc -ne 0 ]; then