Browse code

improved Quantum plugin configuration

The configuration defaults for the openvswitch and linuxbridge plugins
are changing in https://review.openstack.org/#/c/12362/ to address
https://bugs.launchpad.net/quantum/+bug/1045142. To summarize, with no
overriding of default configuration values, tenant networks will now
work on all systems, but are now local to the host. Using GRE tunnels
(openvswitch) or VLANs (openvswitch or linuxbridge) for external
connectivity requires additional configuration. This patch provides
and documents a set of simple shell variables that can be set in
localrc to achieve a range of quantum network configurations.

To use GRE tunnels for remote connectivity with openvswitch, localrc
should include:

Q_PLUGIN=openvswitch
ENABLE_TENANT_TUNNELS=True

Note that OVS GRE tunnels require kernel support that is not in the
Linux kernel source tree, and is not included in all versions of Linux
on which devstack runs.

To use VLANs 1000 through 1999 on eth1 for remote connectivity with
linuxbridge, localrc should include:

Q_PLUGIN=openvswitch
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=1000:1999
PHYSICAL_NETWORK=default
OVS_PHYSICAL_BRIDGE=br-eth1

The OVS bridge br-eth1 must be manually created, and the physical
interface eth1 must be manually added as a port. Any needed host IP
address must be set on br-eth1 rather than eth1. Note that OVS bridges
and ports are persistent.

To use VLANs 1000 through 1999 on eth1 for remote connectivity with
linuxbridge, localrc should include:

Q_PLUGIN=linuxbridge
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=1000:1999
PHYSICAL_NETWORK=default
LB_PHYSICAL_INTERFACE=eth1

The physical interface eth1 must be up, but does not have to have an
IP address. Any existing host IP address configured on eth1 will be
moved to a bridge when the network is activated by the agent, and
moved back when the network is deleted.

Change-Id: I72e9aba1335c55077f4a34495e2d2d9ec1857cd5

Bob Kukura authored on 2012/09/06 04:07:15
Showing 1 changed files
... ...
@@ -456,14 +456,20 @@ FLAT_INTERFACE=${FLAT_INTERFACE:-$GUEST_INTERFACE_DEFAULT}
456 456
 
457 457
 # Using Quantum networking:
458 458
 #
459
-# Make sure that quantum is enabled in ENABLED_SERVICES.  If it is the network
460
-# manager will be set to the QuantumManager.  If you want to run Quantum on
461
-# this host, make sure that q-svc is also in ENABLED_SERVICES.
462
-#
463
-# If you're planning to use the Quantum openvswitch plugin, set Q_PLUGIN to
464
-# "openvswitch" and make sure the q-agt service is enabled in
459
+# Make sure that quantum is enabled in ENABLED_SERVICES.  If you want
460
+# to run Quantum on this host, make sure that q-svc is also in
465 461
 # ENABLED_SERVICES.
466 462
 #
463
+# If you're planning to use the Quantum openvswitch plugin, set
464
+# Q_PLUGIN to "openvswitch" and make sure the q-agt service is enabled
465
+# in ENABLED_SERVICES.  If you're planning to use the Quantum
466
+# linuxbridge plugin, set Q_PLUGIN to "linuxbridge" and make sure the
467
+# q-agt service is enabled in ENABLED_SERVICES.
468
+#
469
+# See "Quantum Network Configuration" below for additional variables
470
+# that must be set in localrc for connectivity across hosts with
471
+# Quantum.
472
+#
467 473
 # With Quantum networking the NET_MAN variable is ignored.
468 474
 
469 475
 
... ...
@@ -713,14 +719,6 @@ EOF
713 713
     install_package mysql-server
714 714
 fi
715 715
 
716
-if is_service_enabled quantum; then
717
-    if [[ "$Q_PLUGIN" = "linuxbridge" ]]; then
718
-        # Install deps
719
-        # FIXME add to files/apts/quantum, but don't install if not needed!
720
-        install_package python-configobj
721
-    fi
722
-fi
723
-
724 716
 if is_service_enabled horizon; then
725 717
     if [[ "$os_PACKAGE" = "deb" ]]; then
726 718
         # Install apache2, which is NOPRIME'd
... ...
@@ -1140,6 +1138,66 @@ fi
1140 1140
 # -------
1141 1141
 
1142 1142
 if is_service_enabled quantum; then
1143
+    #
1144
+    # Quantum Network Configuration
1145
+    #
1146
+    # The following variables control the Quantum openvswitch and
1147
+    # linuxbridge plugins' allocation of tenant networks and
1148
+    # availability of provider networks. If these are not configured
1149
+    # in localrc, tenant networks will be local to the host (with no
1150
+    # remote connectivity), and no physical resources will be
1151
+    # available for the allocation of provider networks.
1152
+
1153
+    # To use GRE tunnels for tenant networks, set to True in
1154
+    # localrc. GRE tunnels are only supported by the openvswitch
1155
+    # plugin, and currently only on Ubuntu.
1156
+    ENABLE_TENANT_TUNNELS=${ENABLE_TENANT_TUNNELS:-False}
1157
+
1158
+    # If using GRE tunnels for tenant networks, specify the range of
1159
+    # tunnel IDs from which tenant networks are allocated. Can be
1160
+    # overriden in localrc in necesssary.
1161
+    TENANT_TUNNEL_RANGES=${TENANT_TUNNEL_RANGE:-1:1000}
1162
+
1163
+    # To use VLANs for tenant networks, set to True in localrc. VLANs
1164
+    # are supported by the openvswitch and linuxbridge plugins, each
1165
+    # requiring additional configuration described below.
1166
+    ENABLE_TENANT_VLANS=${ENABLE_TENANT_VLANS:-False}
1167
+
1168
+    # If using VLANs for tenant networks, set in localrc to specify
1169
+    # the range of VLAN VIDs from which tenant networks are
1170
+    # allocated. An external network switch must be configured to
1171
+    # trunk these VLANs between hosts for multi-host connectivity.
1172
+    #
1173
+    # Example: TENANT_VLAN_RANGE=1000:1999
1174
+    TENANT_VLAN_RANGE=${TENANT_VLAN_RANGE:-}
1175
+
1176
+    # If using VLANs for tenant networks, or if using flat or VLAN
1177
+    # provider networks, set in localrc to the name of the physical
1178
+    # network, and also configure OVS_PHYSICAL_BRIDGE for the
1179
+    # openvswitch agent or LB_PHYSICAL_INTERFACE for the linuxbridge
1180
+    # agent, as described below.
1181
+    #
1182
+    # Example: PHYSICAL_NETWORK=default
1183
+    PHYSICAL_NETWORK=${PHYSICAL_NETWORK:-}
1184
+
1185
+    # With the openvswitch plugin, if using VLANs for tenant networks,
1186
+    # or if using flat or VLAN provider networks, set in localrc to
1187
+    # the name of the OVS bridge to use for the physical network. The
1188
+    # bridge will be created if it does not already exist, but a
1189
+    # physical interface must be manually added to the bridge as a
1190
+    # port for external connectivity.
1191
+    #
1192
+    # Example: OVS_PHYSICAL_BRIDGE=br-eth1
1193
+    OVS_PHYSICAL_BRIDGE=${OVS_PHYSICAL_BRIDGE:-}
1194
+
1195
+    # With the linuxbridge plugin, if using VLANs for tenant networks,
1196
+    # or if using flat or VLAN provider networks, set in localrc to
1197
+    # the name of the network interface to use for the physical
1198
+    # network.
1199
+    #
1200
+    # Example: LB_PHYSICAL_INTERFACE=eth1
1201
+    LB_PHYSICAL_INTERFACE=${LB_PHYSICAL_INTERFACE:-}
1202
+
1143 1203
     # Put config files in ``/etc/quantum`` for everyone to find
1144 1204
     if [[ ! -d /etc/quantum ]]; then
1145 1205
         sudo mkdir -p /etc/quantum
... ...
@@ -1168,22 +1226,6 @@ if is_service_enabled quantum; then
1168 1168
 
1169 1169
     iniset /$Q_PLUGIN_CONF_FILE DATABASE sql_connection mysql:\/\/$MYSQL_USER:$MYSQL_PASSWORD@$MYSQL_HOST\/$Q_DB_NAME?charset=utf8
1170 1170
 
1171
-    OVS_ENABLE_TUNNELING=${OVS_ENABLE_TUNNELING:-True}
1172
-    if [[ "$Q_PLUGIN" = "openvswitch" && "$OVS_ENABLE_TUNNELING" = "True" ]]; then
1173
-        OVS_VERSION=`ovs-vsctl --version | head -n 1 | awk '{print $4;}'`
1174
-        if [ $OVS_VERSION \< "1.4" ] && ! is_service_enabled q-svc ; then
1175
-            echo "You are running OVS version $OVS_VERSION."
1176
-            echo "OVS 1.4+ is required for tunneling between multiple hosts."
1177
-            exit 1
1178
-        fi
1179
-        if [[ "$OVS_DEFAULT_BRIDGE" = "" ]]; then
1180
-            iniset /$Q_PLUGIN_CONF_FILE OVS network_vlan_ranges ""
1181
-        else
1182
-            iniset /$Q_PLUGIN_CONF_FILE OVS network_vlan_ranges default
1183
-        fi
1184
-        iniset /$Q_PLUGIN_CONF_FILE OVS tunnel_id_ranges 1:1000
1185
-    fi
1186
-
1187 1171
     Q_CONF_FILE=/etc/quantum/quantum.conf
1188 1172
     cp $QUANTUM_DIR/etc/quantum.conf $Q_CONF_FILE
1189 1173
 fi
... ...
@@ -1209,33 +1251,96 @@ if is_service_enabled q-svc; then
1209 1209
 
1210 1210
     iniset $Q_CONF_FILE DEFAULT auth_strategy $Q_AUTH_STRATEGY
1211 1211
     quantum_setup_keystone $Q_API_PASTE_FILE filter:authtoken
1212
+
1213
+    # Configure plugin
1214
+    if [[ "$Q_PLUGIN" = "openvswitch" ]]; then
1215
+        if [[ "$ENABLE_TENANT_TUNNELS" = "True" ]]; then
1216
+            iniset /$Q_PLUGIN_CONF_FILE OVS tenant_network_type gre
1217
+            iniset /$Q_PLUGIN_CONF_FILE OVS tunnel_id_ranges $TENANT_TUNNEL_RANGES
1218
+        elif [[ "$ENABLE_TENANT_VLANS" = "True" ]]; then
1219
+            iniset /$Q_PLUGIN_CONF_FILE OVS tenant_network_type vlan
1220
+        else
1221
+            echo "WARNING - The openvswitch plugin is using local tenant networks, with no connectivity between hosts."
1222
+        fi
1223
+
1224
+        # Override OVS_VLAN_RANGES and OVS_BRIDGE_MAPPINGS in localrc
1225
+        # for more complex physical network configurations.
1226
+        if [[ "$OVS_VLAN_RANGES" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then
1227
+            OVS_VLAN_RANGES=$PHYSICAL_NETWORK
1228
+            if [[ "$TENANT_VLAN_RANGE" != "" ]]; then
1229
+                OVS_VLAN_RANGES=$OVS_VLAN_RANGES:$TENANT_VLAN_RANGE
1230
+            fi
1231
+        fi
1232
+        if [[ "$OVS_VLAN_RANGES" != "" ]]; then
1233
+            iniset /$Q_PLUGIN_CONF_FILE OVS network_vlan_ranges $OVS_VLAN_RANGES
1234
+        fi
1235
+    elif [[ "$Q_PLUGIN" = "linuxbridge" ]]; then
1236
+        if [[ "$ENABLE_TENANT_VLANS" = "True" ]]; then
1237
+            iniset /$Q_PLUGIN_CONF_FILE VLANS tenant_network_type vlan
1238
+        else
1239
+            echo "WARNING - The linuxbridge plugin is using local tenant networks, with no connectivity between hosts."
1240
+        fi
1241
+
1242
+        # Override LB_VLAN_RANGES and LB_INTERFACE_MAPPINGS in localrc
1243
+        # for more complex physical network configurations.
1244
+        if [[ "$LB_VLAN_RANGES" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then
1245
+            LB_VLAN_RANGES=$PHYSICAL_NETWORK
1246
+            if [[ "$TENANT_VLAN_RANGE" != "" ]]; then
1247
+                LB_VLAN_RANGES=$LB_VLAN_RANGES:$TENANT_VLAN_RANGE
1248
+            fi
1249
+        fi
1250
+        if [[ "$LB_VLAN_RANGES" != "" ]]; then
1251
+            iniset /$Q_PLUGIN_CONF_FILE VLANS network_vlan_ranges $LB_VLAN_RANGES
1252
+        fi
1253
+    fi
1212 1254
 fi
1213 1255
 
1214 1256
 # Quantum agent (for compute nodes)
1215 1257
 if is_service_enabled q-agt; then
1258
+    # Configure agent for plugin
1216 1259
     if [[ "$Q_PLUGIN" = "openvswitch" ]]; then
1217
-        # Set up integration bridge
1260
+        # Setup integration bridge
1218 1261
         OVS_BRIDGE=${OVS_BRIDGE:-br-int}
1219 1262
         quantum_setup_ovs_bridge $OVS_BRIDGE
1220
-        if [[ "$OVS_ENABLE_TUNNELING" == "True" ]]; then
1263
+
1264
+        # Setup agent for tunneling
1265
+        if [[ "$ENABLE_TENANT_TUNNELS" = "True" ]]; then
1266
+            # Verify tunnels are supported
1267
+            # REVISIT - also check kernel module support for GRE and patch ports
1268
+            OVS_VERSION=`ovs-vsctl --version | head -n 1 | awk '{print $4;}'`
1269
+            if [ $OVS_VERSION \< "1.4" ] && ! is_service_enabled q-svc ; then
1270
+                echo "You are running OVS version $OVS_VERSION."
1271
+                echo "OVS 1.4+ is required for tunneling between multiple hosts."
1272
+                exit 1
1273
+            fi
1221 1274
             iniset /$Q_PLUGIN_CONF_FILE OVS local_ip $HOST_IP
1222
-        else
1223
-            # Need bridge if not tunneling
1224
-            OVS_DEFAULT_BRIDGE=${OVS_DEFAULT_BRIDGE:-br-$GUEST_INTERFACE_DEFAULT}
1225 1275
         fi
1226
-        if [[ "$OVS_DEFAULT_BRIDGE" = "" ]]; then
1227
-            iniset /$Q_PLUGIN_CONF_FILE OVS bridge_mappings ""
1228
-        else
1276
+
1277
+        # Setup physical network bridge mappings.  Override
1278
+        # OVS_VLAN_RANGES and OVS_BRIDGE_MAPPINGS in localrc for more
1279
+        # complex physical network configurations.
1280
+        if [[ "$OVS_BRIDGE_MAPPINGS" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$OVS_PHYSICAL_BRIDGE" != "" ]]; then
1281
+            OVS_BRIDGE_MAPPINGS=$PHYSICAL_NETWORK:$OVS_PHYSICAL_BRIDGE
1282
+
1229 1283
             # Configure bridge manually with physical interface as port for multi-node
1230
-            sudo ovs-vsctl --no-wait -- --may-exist add-br $OVS_DEFAULT_BRIDGE
1231
-            iniset /$Q_PLUGIN_CONF_FILE OVS bridge_mappings default:$OVS_DEFAULT_BRIDGE
1284
+            sudo ovs-vsctl --no-wait -- --may-exist add-br $OVS_PHYSICAL_BRIDGE
1285
+        fi
1286
+        if [[ "$OVS_BRIDGE_MAPPINGS" != "" ]]; then
1287
+            iniset /$Q_PLUGIN_CONF_FILE OVS bridge_mappings $OVS_BRIDGE_MAPPINGS
1232 1288
         fi
1289
+
1233 1290
         AGENT_BINARY="$QUANTUM_DIR/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py"
1234 1291
     elif [[ "$Q_PLUGIN" = "linuxbridge" ]]; then
1235
-       # Start up the quantum <-> linuxbridge agent
1236
-       # set the default network interface
1237
-       QUANTUM_LB_PRIVATE_INTERFACE=${QUANTUM_LB_PRIVATE_INTERFACE:-$GUEST_INTERFACE_DEFAULT}
1238
-       iniset /$Q_PLUGIN_CONF_FILE LINUX_BRIDGE physical_interface_mappings default:$QUANTUM_LB_PRIVATE_INTERFACE
1292
+        # Setup physical network interface mappings.  Override
1293
+        # LB_VLAN_RANGES and LB_INTERFACE_MAPPINGS in localrc for more
1294
+        # complex physical network configurations.
1295
+        if [[ "$LB_INTERFACE_MAPPINGS" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$LB_PHYSICAL_INTERFACE" != "" ]]; then
1296
+            LB_INTERFACE_MAPPINGS=$PHYSICAL_NETWORK:$LB_PHYSICAL_INTERFACE
1297
+        fi
1298
+        if [[ "$LB_INTERFACE_MAPPINGS" != "" ]]; then
1299
+            iniset /$Q_PLUGIN_CONF_FILE LINUX_BRIDGE physical_interface_mappings $LB_INTERFACE_MAPPINGS
1300
+        fi
1301
+
1239 1302
        AGENT_BINARY="$QUANTUM_DIR/quantum/plugins/linuxbridge/agent/linuxbridge_quantum_agent.py"
1240 1303
     fi
1241 1304
 fi
... ...
@@ -2175,13 +2280,6 @@ if is_service_enabled q-svc; then
2175 2175
         fi
2176 2176
    fi
2177 2177
 
2178
-   # Start up the quantum agent
2179
-   screen_it q-agt "sudo python $AGENT_BINARY --config-file $Q_CONF_FILE --config-file /$Q_PLUGIN_CONF_FILE"
2180
-   # Start up the quantum dhcp agent
2181
-   screen_it q-dhcp "sudo python $AGENT_DHCP_BINARY --config-file $Q_CONF_FILE --config-file=$Q_DHCP_CONF_FILE"
2182
-   # Start up the quantum l3 agent
2183
-   screen_it q-l3 "sudo python $AGENT_L3_BINARY --config-file $Q_CONF_FILE --config-file=$Q_L3_CONF_FILE"
2184
-
2185 2178
 elif is_service_enabled mysql && is_service_enabled nova; then
2186 2179
     # Create a small network
2187 2180
     $NOVA_BIN_DIR/nova-manage network create private $FIXED_RANGE 1 $FIXED_NETWORK_SIZE $NETWORK_CREATE_ARGS
... ...
@@ -2193,6 +2291,11 @@ elif is_service_enabled mysql && is_service_enabled nova; then
2193 2193
     $NOVA_BIN_DIR/nova-manage floating create --ip_range=$TEST_FLOATING_RANGE --pool=$TEST_FLOATING_POOL
2194 2194
 fi
2195 2195
 
2196
+# Start up the quantum agents if enabled
2197
+screen_it q-agt "sudo python $AGENT_BINARY --config-file $Q_CONF_FILE --config-file /$Q_PLUGIN_CONF_FILE"
2198
+screen_it q-dhcp "sudo python $AGENT_DHCP_BINARY --config-file $Q_CONF_FILE --config-file=$Q_DHCP_CONF_FILE"
2199
+screen_it q-l3 "sudo python $AGENT_L3_BINARY --config-file $Q_CONF_FILE --config-file=$Q_L3_CONF_FILE"
2200
+
2196 2201
 # The group **libvirtd** is added to the current user in this script.
2197 2202
 # Use 'sg' to execute nova-compute as a member of the **libvirtd** group.
2198 2203
 # ``screen_it`` checks ``is_service_enabled``, it is not needed here