Controller Nodes

• OpenStack services, such as nova-api, glance-api, keystone-api, neutron-api, nova-scheduler, cinder-api are stateless services that do not require any special attention besides load balancing.
• Horizon, as a typical web application, requires sticky sessions to be enabled at the load balancer.
• RabbitMQ provides active/active high availability using mirrored queues and is deployed with custom resource agent scripts for Pacemaker.
• MySQL high availability is achieved through Galera deployment and custom resource agent scripts for Pacemaker. Please, note that HAProxy configures MySQL backends as active/passive because OpenStack support for multi-node writes to Galera nodes is not production ready yet.
• Neutron agents are active/passive and are managed by custom resource agent scripts for Pacemaker.
• Ceph monitors implement their own quorum based HA mechanism and require time synchronization between all nodes. Clock drift higher than 50ms may break the quorum or even crash the Ceph service.

Compute Nodes

Storage Nodes

Corosync Settings

totem {
  version:                             2
  token:                               3000
  token_retransmits_before_loss_const: 10
  join:                                60
  consensus:                           3600
  vsftype:                             none
  max_messages:                        20
  clear_node_high_bit:                 yes
  rrp_mode:                            none
  secauth:                             off
  threads:                             0
  interface {
    ringnumber:  0
    bindnetaddr: 10.107.0.8
    mcastaddr:   239.1.1.2
    mcastport:   5405
  }
}

Pacemaker Settings

How Fuel Deploys HA

# crm configure show

Dual hypervisor support

Multi-node HA Deployment with vSphere integration

Example of network topology

Fuel running under vSphere

Logical Networks

The Fuel Master node uses this network to provision and orchestrate the OpenStack environment. It is used during installation to provide DNS, DHCP, and gateway services to a node before that node is provisioned. Nodes retrieve their network configuration from the Fuel Master node using DHCP, which is why this network must be isolated from the rest of your network and must not have a DHCP server other than the Fuel Master running on it.

The word “Public” means that these addresses can be used to communicate with the cluster and its VMs from outside of the cluster (the Internet, corporate network, end users).

The public network provides connectivity to the globally routed address space for VMs. The IP address from the public network that has been assigned to a compute node is used as the source for the Source NAT performed for traffic going from VM instances on the compute node to the Internet.

The public network also provides Virtual IPs for public endpoints, which are used to connect to OpenStack services APIs.

Finally, the public network provides a contiguous address range for the floating IPs that are assigned to individual VM instances by the project administrator. Nova Network or Neutron services can then configure this address on the public network interface of the Network controller node. Environments based on Nova Network use iptables to create a Destination NAT from this address to the private IP of the corresponding VM instance through the appropriate virtual bridge interface on the Network controller node.

For security reasons, the public network is usually isolated from other networks in the cluster.

If you use tagged networks for your configuration and combine multiple networks onto one NIC, you should leave the Public network untagged on that NIC. This is not a requirement, but it simplifies external access to OpenStack Dashboard and public OpenStack API endpoints.

Part of a cluster’s internal network. It carries replication traffic from Ceph or Swift. Ceph public traffic is dispatched through br-mgmt bridge (Management network).

Part of the cluster’s internal network. It is used to put tagged VLAN traffic from private tenant networks on physical NIC interfaces. This network can also be used for serving iSCSI protocol exchanges between Compute and Storage nodes. As to the Management, it serves for all other internal communications, including database queries, AMQP messaging, high availability services).

The private network facilitates communication between each tenant’s VMs. Private network address spaces are not a part of the enterprise network address space; fixed IPs of virtual instances cannot be accessed directly from the rest of the Enterprise network.

Just like the public network, the private network should be isolated from other networks in the cluster for security reasons.

The internal network connects all OpenStack nodes in the environment. All components of an OpenStack environment communicate with each other using this network. This network must be isolated from both the private and public networks for security reasons. The internal network can also be used for serving iSCSI protocol exchanges between Compute and Storage nodes. The Internal Network is a generalizing term; it means that any network except for Public can be regarded as Internal: for example, Storage or Management. Do not confuse Internal with Private, as the latter is only related to the networks within a tenant, that provides communication between VMs within the specific tenant.

HA deployment for Networking

Implementing Multiple Cluster Networks

• Each of the major logical network (public, management, storage, and fuelweb_admin) is associated with a Node Group rather than a cluster.
• Each Node Group belongs to a cluster.
• A default Node Group is created for each cluster. The default values are derived from Fuel Menu (for the Fuel Admin (PXE) network) and release metadata.
• Each cluster can support multiple Node Groups.

• A node serializes its network information based on its relationship to networks in its Node Group.
• Each node must have a Node Group; if it is not explicitly assigned to one, it is assumed to be a member of the default network group. If it is not configured properly, it will result in the cluster failing to deploy.
• A set of default networks is generated when a Node Group is created. These networks are deleted when the Node Group is deleted.
• Each logical network is associated with a Node Group rather than with a cluster.
• Each fuelweb_admin network must have a DHCP network configured in the dnsmasq.template file.
• DHCP requests can be forwarded to the Fuel Master node using either of the following methods:
  • configure switches to relay DHCP
  • using a relay client such as dhcp-helper

[root@nailgun ~]# fuel nodegroup

id | cluster | name
---|---------|---------------
1  | 1       | default
2  | 1       | alpha

Public and Floating IP address requirements

Router

• Use the default routing via a router in the Public network
• Use the the management network to access your management infrastructure (L3 connectivity if necessary)
• The Storage and VM networks should be configured without access to other networks (no L3 connectivity)

Switches

• Neutron Switch configuration (Cisco Catalyst 2960G)
• Neutron Switch configuration (Juniper EX4200)
• Nova-network Switch configuration (Cisco Catalyst 2960G)
• Nova-network Switch configuration (Juniper EX4200)

• Configure all access ports to allow non-tagged PXE booting connections from each slave node to the Fuel Master node. This network is referred to as the Fuel network.
• By default, the Fuel Master node uses the eth0 interface to serve PXE requests on this network, but this can be changed during installation of the Fuel Master node.
• If you use the eth0 interface for PXE requests, you must set the switch port for eth0 on the Fuel Master node to access mode.
• We recommend that you use the eth0 interfaces of all other nodes for PXE booting as well. Corresponding ports must also be in access mode.
• Taking into account that this is the network for PXE booting, do not mix this L2 segment with any other network segments. Fuel runs a DHCP server, and, if there is another DHCP on the same L2 network segment, both the company’s infrastructure and Fuel’s are unable to function properly.
• You must also configure each of the switch’s ports connected to nodes as an “STP Edge port” (or a “spanning-tree port fast trunk”, according to Cisco terminology). If you do not do that, DHCP timeout issues may occur.

interface GigabitEthernet0/6               # switch port
description s0_eth0 jv                     # description
switchport trunk encapsulation dot1q       # enables VLANs
switchport trunk native vlan 262           # access port, untags VLAN 262
switchport trunk allowed vlan 100,102,104  # 100,102,104 VLANs are passed with tags
switchport mode trunk                      # To allow more than 1 VLAN on the port
spanning-tree portfast trunk               # STP Edge port to skip network loop
                                           # checks (to prevent DHCP timeout issues)
vlan 262,100,102,104                       # Might be needed for enabling VLANs

Neutron versus Nova-Network

• VLAN segmentation Ideally, “Private network” traffic is located on a dedicated network adapter that is attached to an untagged network port. It is, however, possible for this network to share a network adapter with other networks. In this case, you should use non-intersecting VLAN-ID ranges for “Private network” and other networks.
• GRE segmentation In this mode of operation, Neutron does not require a dedicated network adapter. Neutron builds a mesh of GRE tunnels from each compute node and controller nodes to every other node. Private networks for each tenant make use of this mesh for isolated traffic.

Neutron with VLAN segmentation and OVS

Neutron with GRE segmentation and OVS

Known limitations

• Neutron will not allocate a floating IP range for your tenants. After each tenant is created, a floating IP range must be created. Note that this does not prevent Internet connectivity for a tenant’s instances, but it would prevent them from receiving incoming connections. You, the administrator, should assign a floating IP addresses for the tenant. Below are steps you can follow to do this:

  # get admin credentials:
  source /root/openrc
  # get admin tenant-ID:
  keystone tenant-list
  

  id	name	enabled
  b796f91df6b84860a7cd474148fb2229	admin	True
  cba7b0ff68ee4985816ac3585c8e23a9	services	True

  # create one floating-ip address for admin tenant:
  neutron floatingip-create --tenant-id=b796f91df6b84860a7cd474148fb2229 net04_ext
  

• You can’t combine Private or Admin network with any other networks on one NIC.
• To deploy OpenStack using Neutron with GRE segmentation, each node requires at least 2 NICs.
• To deploy OpenStack using Neutron with VLAN segmentation, each node requires at least 3 NICs.

NIC Assignment Example (Neutron VLAN)

eth0:

The Admin (PXE) network, used for communication with Fuel Master for deployment.

eth1:

The public network and floating IPs assigned to VMs

eth2:

The private network, for communication between OpenStack VMs, and the bridge interface (VLANs)

Nova-network FlatDHCP Manager

Nova-network VLAN Manager

Fuel Deployment Schema

Configuring the network

• Admin (Fuel) network: untagged on the scheme
• Public network: VLAN 101
• Floating network: VLAN 101
• Management network: VLAN 100
• Storage network: VLAN 102
• Fixed network: VLANs 103-200

Nova-network Planning Examples

Nova-network FlatDHCP

• eth0 – VLAN tagged port for networks: Storage, Public/Floating, Private, Management and Administrative (untagged)

• eth0 – Management network (tagged), Storage network (tagged) and Administrative network  (untagged)
• eth1 – VLAN tagged port with VLANs for networks: Public/Floating, Private

• eth0 – untagged port for Administrative network
• eth1 – VLAN tagged port with VLANs for networks: Public/Floating, Private, Management
• eth2 – untagged port for Storage network

• eth0 – untagged port for Administrative network
• eth1 – tagged port for networks: Public/Floating, Management
• eth2 – untagged port for Private network
• eth3 – untagged port for Storage network

• Use the default routing via a router in the Public network
• Use the the management network to access to your management infrastructure (L3 connectivity if necessary)
• The administrative network or only the Fuel server (via dedicated NIC) should have Internet access
• The Storage and Private network (VLANs) should be configured without access to other networks (no L3 connectivity)

Nova-network VLAN Manager

• eth0 – VLAN tagged port for networks: Storage, Public/Floating, Private  (where the number of VLANs depends on the number of tenant networks with a continuous range), Management and Administrative network (untagged)

• eth0 – Management network (tagged), Storage network (tagged) and Administrative network  (untagged)
• eth1 – VLAN tagged port with minimum two VLANs for networks: Public/Floating, Private (where number of VLANs depend on number of tenant networks – continuous range)

• eth0 – untagged port for Administrative network
• eth1 – VLAN tagged port with two VLANs for networks: Public/Floating, Management Private (where the number of VLANs depends on the number of tenant networks with a continuous range)
• eth2 – untagged port for Storage network

• eth0 – untagged port for Administrative network
• eth1 – tagged port for networks: Public/Floating, Management
• eth2 – VLAN tagged port for Private network, with defined VLAN range IDs – continuous range
• eth3 – untagged port for Storage network

• Use the default routing via a router in the Public network
• Use the the management network to access to your management infrastructure (L3 connectivity if necessary)
• The administrative network or only the Fuel server (via dedicated NIC) should have Internet access
• The Storage and Private network (VLANs) should be configured without access to other networks (no L3 connectivity)

Reference Network Model in Neutron

• Create an OVS bridge for each NIC except for the NIC with Admin network (for example, br-eth0 bridge for eth0 NIC) and put NICs into their bridges
• Create a separate bridge for each OpenStack network:
  • br-ex for the Public network
  • br-prv for the Private network
  • br-mgmt for the Management network
  • br-storage for the Storage network
• Connect each network’s bridge with an appropriate NIC bridge using an OVS patch with an appropriate VLAN tag.
• Assign network IP addresses to the corresponding bridges.

Adjust the Network Configuration via CLI

[root@fuel ~]# fuel --env 1 deployment default
directory /root/deployment_1 was created
Created /root/deployment_1/compute_1.yaml
Created /root/deployment_1/controller_2.yaml
[root@fuel ~]# vi ./deployment_1/compute_1.yaml
[root@fuel ~]# fuel --env 1 deployment --upload

• interfaces – A hash of NICs and their low-level/physical parameters. You can set an MTU feature here.
• provider – Set to ‘ovs’ for Neutron.
• endpoints – A hash of network ports (OVS ports or NICs) and their IP settings.
• roles – A hash that specifies the mappings between the endpoints and internally-used roles in Puppet manifests (‘management’, ‘storage’, and so on).
• transformations – An ordered list of OVS network primitives.

network_scheme:
 endpoints:
   br-ex:
     IP:
     - 172.16.0.7/24
     gateway: 172.16.0.1
   br-fw-admin:
     IP:
     - 10.20.0.7/24
   br-mgmt:
     IP:
     - 192.168.0.7/24
   br-prv:
     IP: none
   br-storage:
     IP:
     - 192.168.1.6/24
 interfaces:
   eth0:
     mtu: 1234
     L2:
       vlan_splinters: 'off'
   eth1:
     mtu: 4321
     L2:
       vlan_splinters: 'off'
   eth2:
     L2:
       vlan_splinters: 'off'

The “Transformations” Section

• add-br – To add an OVS bridge to the system
• add-port – To add a port to an existent OVS bridge
• add-bond – To create a port in OVS bridge and add aggregated NICs to it
• add-patch – To create an OVS patch between two existing OVS bridges

{
  "action": "add-br",         # type of primitive
  "name": "xxx"               # unique name of the new bridge
},
{
  "action": "add-port",       # type of primitive
  "name": "xxx-port",         # unique name of the new port
  "bridge": "xxx",            # name of the bridge where the port should be created
  "type": "internal",         # [optional; default: "internal"] a type of OVS
                              # interface # for the port (see OVS documentation);
                              # possible values:
                              # "system", "internal", "tap", "gre", "null"
  "tag": 0,                   # [optional; default: 0] a 802.1q tag of traffic that
                              # should be captured from an OVS bridge;
                              # possible values: 0 (means port is a trunk),
                              # 1-4094 (means port is an access)
  "trunks": [],               # [optional; default: []] a set of 802.1q tags
                              # (integers from 0 to 4095) that are allowed to
                              # pass through if "tag" option equals 0;
                              # possible values: an empty list (all traffic passes),
                              # 0 (untagged traffic only), 1 (strange behaviour;
                              # shouldn't be used), 2-4095 (traffic with this
                              # tag passes); e.g. [0,10,20]
  "port_properties": [],      # [optional; default: []] a list of additional
                              # OVS port properties to modify them in OVS DB
  "interface_properties": [], # [optional; default: []] a list of additional
                              # OVS interface properties to modify them in OVS DB
},
{
  "action": "add-bond",       # type of primitive
  "name": "xxx-port",         # unique name of the new bond
  "interfaces": [],           # a set of two or more bonded interfaces' names;
                              # e.g. ['eth1','eth2']
  "bridge": "xxx",            # name of the bridge where the bond should be created
  "tag": 0,                   # [optional; default: 0] a 802.1q tag of traffic which
                              # should be catched from an OVS bridge;
                              # possible values: 0 (means port is a trunk),
                              # 1-4094 (means port is an access)
  "trunks": [],               # [optional; default: []] a set of 802.1q tags
                              # (integers from 0 to 4095) which are allowed to
                              # pass through if "tag" option equals 0;
                              # possible values: an empty list (all traffic passes),
                              # 0 (untagged traffic only), 1 (strange behaviour;
                              # shouldn't be used), 2-4095 (traffic with this
                              # tag passes); e.g. [0,10,20]
  "properties": [],           # [optional; default: []] a list of additional
                              # OVS bonded port properties to modify them in OVS DB;
                              # you can use it to set the aggregation mode and
                              # balancing # strategy, to configure LACP, and so on
                              # (see the OVS documentation)
},
{
  "action": "add-patch",      # type of primitive
  "bridges": ["br0", "br1"],  # a pair of different bridges that will be connected
  "peers": ["p1", "p2"],      # [optional] abstract names for each end of the patch
  "tags": [0, 0] ,            # [optional; default: [0,0]] a pair of integers that
                              # represent an 802.1q tag of traffic that is
                              # captured from an appropriate OVS bridge; possible
                              # values: 0 (means port is a trunk), 1-4094 (means
                              # port is an access)
  "trunks": [],               # [optional; default: []] a set of 802.1q tags
                              # (integers from 0 to 4095) which are allowed to
                              # pass through each bridge if "tag" option equals 0;
                              # possible values: an empty list (all traffic passes),
                              # 0 (untagged traffic only), 1 (strange behavior;
                              # shouldn't be used), 2-4095 (traffic with this
                              # tag passes); e.g., [0,10,20]
}

Types of Bonding

• Balance-rr – Round-robin policy. This mode provides load balancing and fault tolerance.
• Active-backup – Active-backup policy: Only one slave in the bond is active.This mode provides fault tolerance.
• Balance-xor – XOR policy: Transmit based on the selected transmit hash policy. This mode provides load balancing and fault tolerance.
• Broadcast – Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
• balance-tlb – Adaptive transmit load balancing based on a current links’ utilization. This mode provides load balancing and fault tolerance.
• balance-alb – Adaptive transmit and receive load balancing based on the current links’ utilization. This mode provides load balancing and fault tolerance.
• balance-slb – Modification of balance-alb mode. SLB bonding allows a limited form of load balancing without the remote switch’s knowledge or cooperation. SLB assigns each source MAC+VLAN pair to a link and transmits all packets from that MAC+VLAN through that link. Learning in the remote switch causes it to send packets to that MAC+VLAN through the same link.
• balance-tcp – Adaptive transmit load balancing among interfaces.

• Layer2 – Uses XOR of hardware MAC addresses to generate the hash.
• Layer2+3 – uses a combination of layer2 and layer3 protocol information to generate the hash.
• Layer3+4 – uses upper layer protocol information, when available, to generate the hash.
• Encap2+3 – uses the same formula as layer2+3 but it relies on skb_flow_dissect to obtain the header fields which might result in the use of inner headers if an encapsulation protocol is used. For example this will improve the performance for tunnel users because the packets will be distributed according to the encapsulated flows.
• Encap3+4 – Similar to Encap2+3 but uses layer3+4.

An Example of NIC Aggregation using Fuel CLI tools

• Create a separate OVS bridge “br-bond0” instead of “br-eth2” and “br-eth3”.
• Connect “eth2” and “eth3” to “br-bond0” as a bonded port with property “lacp=active”.
• Connect “br-prv” and “br-storage” bridges to “br-bond0” by OVS patches.
• Leave all of the other things unchanged.

'network_scheme':
  'provider': 'ovs'
  'version': '1.0'
  'interfaces':
    'eth0': {}
    'eth1': {}
    'eth2': {}
    'eth3': {}
  'endpoints':
    'br-ex':
      'IP': ['172.16.0.2/24']
      'gateway': '172.16.0.1'
    'br-mgmt':
      'IP': ['192.168.0.2/24']
    'br-prv': {'IP': 'none'}
    'br-storage':
      'IP': ['192.168.1.2/24']
    'eth0':
      'IP': ['10.20.0.4/24']
  'roles':
    'ex': 'br-ex'
    'fw-admin': 'eth0'
    'management': 'br-mgmt'
    'private': 'br-prv'
    'storage': 'br-storage'
  'transformations':
  - 'action': 'add-br'
    'name': 'br-ex'
  - 'action': 'add-br'
    'name': 'br-mgmt'
  - 'action': 'add-br'
    'name': 'br-storage'
  - 'action': 'add-br'
    'name': 'br-prv'
  - 'action': 'add-br'
    'name': 'br-bond0'
  - 'action': 'add-br'
    'name': 'br-eth1'
  - 'action': 'add-bond'
    'bridge': 'br-bond0'
    'interfaces': ['eth2', 'eth3']
    'properties': ['lacp=active']
    'name': 'bond0'
  - 'action': 'add-port'
    'bridge': 'br-eth1'
    'name': 'eth1'
  - 'action': 'add-patch'
    'bridges': ['br-bond0', 'br-storage']
    'tags': [103, 0]
  - 'action': 'add-patch'
    'bridges': ['br-eth1', 'br-ex']
    'tags': [101, 0]
  - 'action': 'add-patch'
    'bridges': ['br-eth1', 'br-mgmt']
    'tags': [102, 0]
  - 'action': 'add-patch'
    'bridges': ['br-bond0', 'br-prv']

• Create a new interface “bond0” instead of “br-eth2” and “br-eth3”.
• Connect “eth2” and “eth3” to “bond0” as a bonded port.
• Add ‘provider’: ‘lnx’ to choose Linux native mode.
• Add properties as a hash instead of an array used in ovs mode. Properties are same as options used during the bonding kernel modules loading. You should provide which mode this bonding interface should use. Any other options are not mandatory. You can find all these options in the Linux Kernel Documentation.

  ‘properties’:

  ‘mode’: 1

• Connect “br-prv” and “br-storage” bridges to “br-bond0” by OVS patches.
• Leave all of the other things unchanged.

'network_scheme':
  'provider': 'ovs'
  'version': '1.0'
  'interfaces':
    'eth0': {}
    'eth1': {}
    'eth2': {}
    'eth3': {}
  'endpoints':
    'br-ex':
      'IP': ['172.16.0.2/24']
      'gateway': '172.16.0.1'
    'br-mgmt':
      'IP': ['192.168.0.2/24']
    'br-prv': {'IP': 'none'}
    'br-storage':
      'IP': ['192.168.1.2/24']
    'eth0':
      'IP': ['10.20.0.4/24']
  'roles':
    'ex': 'br-ex'
    'fw-admin': 'eth0'
    'management': 'br-mgmt'
    'private': 'br-prv'
    'storage': 'br-storage'
  'transformations':
  - 'action': 'add-br'
    'name': 'br-ex'
  - 'action': 'add-br'
    'name': 'br-mgmt'
  - 'action': 'add-br'
    'name': 'br-storage'
  - 'action': 'add-br'
    'name': 'br-prv'
  - 'action': 'add-br'
    'name': 'br-bond0'
  - 'action': 'add-br'
    'name': 'br-eth1'
  - 'action': 'add-bond'
    'bridge': 'br-bond0'
    'interfaces': ['eth2', 'eth3']
    'provider': 'lnx'
    'properties':
      'mode': '1'
    'name': 'bond0'
  - 'action': 'add-port'
    'bridge': 'br-eth1'
    'name': 'eth1'
  - 'action': 'add-patch'
    'bridges': ['br-bond0', 'br-storage']
    'tags': [103, 0]
  - 'action': 'add-patch'
    'bridges': ['br-eth1', 'br-ex']
    'tags': [101, 0]
  - 'action': 'add-patch'
    'bridges': ['br-eth1', 'br-mgmt']
    'tags': [102, 0]
  - 'action': 'add-patch'
    'bridges': ['br-bond0', 'br-prv']

Viewing the control files on the Fuel Master node

cd /var/www/nailgun/bootstrap
mkdir initramfs
cd initramfs
gunzip -c ../initramfs.img | cpio -idv

cat /etc/fuel-agent/fuel-agent.conf

Troubleshooting image-based provisioning

• Bootstrap
  • etc/fuel-agent/fuel-agent.conf — main configuration file for the Fuel Agent, defines the location of the provision data file, data format and log output, whether debugging is on or off, and so forth.
  • tmp/provision.json — Astute puts this file on a node (on the in-memory file system) just before running the provision script.
  • usr/bin/provision — executable entry point for provisioning. Astute runs this; it can also be run manually.
• Master
  • var/log/remote/node-N.domain.tld/bootstrap/fuel-agent.log — this is where Fuel Agent log messages are recorded when the provision script is run; <N> is the nodeID of the provisioned node.

Task schema

- id: graph_node_id
  type: one of [stage, group, skipped, puppet, shell etc]
  role: [match where this tasks should be executed]
  requires: [requirements for a specific node]
  required_for: [specify which nodes depend on this task]

Stages

- pre_deployment_start
- pre_deployment_end
- deploy_start
- deploy_end
- post_deployment_start
- post_deployment_end

- id: deploy_end
  type: stage
  requires: [deploy_start]

Groups

- id: controller
  type: group
  role: [controller]
  requires: [primary-controller]
  required_for: [deploy_end]
  parameters:
    strategy:
      type: parallel
        amount: 6

Skipped

- id: netconfig
  type: skipped
  groups: [primary-controller, controller, cinder, compute, ceph-osd,
           zabbix-server, primary-mongo, mongo]
  required_for: [deploy_end]
  requires: [logging]
  parameters:
    puppet_manifest: /etc/puppet/modules/osnailyfacter/other_path/netconfig.pp
    puppet_modules: /etc/puppet/modules
    timeout: 3600

Puppet

- id: netconfig
    type: puppet
    groups: [primary-controller, controller, cinder, compute, ceph-osd,
             zabbix-server, primary-mongo, mongo]
    required_for: [deploy_end]
    requires: [logging]
    parameters:
      puppet_manifest: /etc/puppet/modules/osnailyfacter/other_path/netconfig.pp
      puppet_modules: /etc/puppet/modules
      timeout: 3600

Shell

- id: enable_quorum
  type: shell
  role: [primary-controller]
  requires: [post_deployment_start]
  required_for: [post_deployment_end]
  parameters:
    cmd: ruby /etc/puppet/modules/osnailyfacter/modular/astute/enable_quorum.rb
    timeout: 180

Upload file

- id: upload_data_to_file
  type: upload_file
  role: '*'
  requires: [pre_deployment_start]
  parameters:
    path: /etc/file_name
    data: 'arbitrary info'

Sync

- id: rsync_core_puppet
  type: sync
  role: '*'
  required_for: [pre_deployment_end]
  requires: [upload_core_repos]
  parameters:
    src: rsync://<FUEL_MASTER_IP>:/puppet/
    dst: /etc/puppet
    timeout:

Copy files

- id: copy_keys
  type: copy_files
  role: '*'
  required_for: [pre_deployment_end]
  requires: [generate_keys]
  parameters:
    files:
      - src: /var/lib/fuel/keys/{CLUSTER_ID}/neutron/neutron.pub
        dst: /var/lib/astute/neutron/neutron.pub
    permissions: '0600'
    dir_permissions: '0700'

API

fuel rel --sync-deployment-tasks --dir /etc/puppet

fuel rel --rel <id> --deployment-tasks --download
fuel rel --rel <id> --deployment-tasks --upload

fuel env --env <id> --deployment-tasks --download
fuel env --env <id> --deployment-tasks --upload

fuel node --node <1>,<2>,<3> --tasks upload_repos netconfig

Skipping tasks

fuel node --node <1>,<2>,<3> --skip netconfig hiera

fuel node --node <1>,<2>,<3> --end netconfig

fuel node --node <1>,<2>,<3> --start netconfig

fuel node --node <1>,<2>,<3> --start netconfig --skip netconfig

fuel node --node <1>,<2>,<3> --start netconfig --end upload_cirros

Graph representation

FAQ

fuel node --node <1>,<2>,<3> --end netconfig

Additional task for an existing role

1. Add the task description to /etc/puppet/2014.2.2-6.1/modules/my_tasks.yaml file.

   - id: my_task
   type: puppet
   groups: [compute]
   required_for: [deploy_end]
   requires: [netconfig]
   parameters:
      puppet_manifest: /etc/puppet/modules/my_task.pp
      puppet_modules: /etc/puppet/modules
      timeout: 3600
   

2. Run the following command:

   fuel rel --sync-deployment-tasks --dir /etc/puppet/2014.2.2-6.1
   

Skipping task by API or by configuration

• Change the task’s type to skipped:

  - id: horizon
  type: skipped
  role: [primary-controller]
  requires: [post_deployment_start]
  required_for: [post_deployment_end]
  

• Add a condition that is always false:

   - id: horizon
  type: puppet
  role: [primary-controller]
  requires: [post_deployment_start]
  required_for: [post_deployment_end]
  condition: 'true != false'
  

• Do an API request:

  fuel node --node <1>,<2>,<3> --skip horizon
  

Creating a separate role and attaching a task to it

1. Create a file with redis.yaml with the following content:

   meta:
     description: Simple redis server
     name: Controller
   name: redis
   volumes_roles_mapping:
     - allocate_size: min
       id: os
   

2. Create a role:

   fuel role --rel 1 --create --file redis.yaml
   

3. After this is done, you can go to the Fuel web UI and check if a role redis is created.
4. You can now attach tasks to the role. First, install redis puppet module:

   puppet module install thomasvandoren-redis
   

5. Write a simple manifest to /etc/puppet/modules/redis/example/simple_redis.pp and include redis.
6. Create a configuration for Fuel in /etc/puppet/modules/redis/example/redis_tasks.yaml:

   # redis group
     - id: redis
       type: group
       role: [redis]
       required_for: [deploy_end]
       tasks: [globals, hiera, netconfig, install_redis]
       parameters:
         strategy:
           type: parallel
   
   # Install simple redis server
     - id: install_redis
       type: puppet
       requires: [netconfig]
       required_for: [deploy_end]
       parameters:
         puppet_manifest: /etc/puppet/modules/redis/example/simple_redis.pp
         puppet_modules: /etc/puppet/modules
         timeout: 180
   

7. Run the following command:

   fuel rel --sync-deployment-tasks --dir /etc/puppet/2014.2.2-6.1/
   

8. Create an enviroment. Note the following:
   • configure public network properly since redis packages are fetched from the upstream.
   • enable Assign public network to all nodes option on the Settings tab of the Fuel web UI.
9. Provision the redis node:

   fuel node --node <1> --env <1> --provision
   

10. Finish the installation on install_redis (there is no need to execute all tasks from post_deployment stage):

    fuel node --node <1> --end install_redis
    

Swapping a task with a custom task

- id: netconfig
  type: puppet
  groups: [primary-controller, controller, cinder, compute, ceph-osd, zabbix-server, primary-mongo, mongo]
  required_for: [deploy_end]
  requires: [logging]
  parameters:
      # old puppet manifest
      # puppet_manifest: /etc/puppet/modules/osnailyfacter/netconfig.pp

      puppet manifest: /etc/puppet/modules/osnailyfacter/custom_network_configuration.pp
      puppet_modules: /etc/puppet/modules
      timeout: 3600

Containers structure

Architecture in 6.1 as compared to 6.0

Mirantis OpenStack 6.1 Network Performance Hardware

• Compute nodes:
  • Node is a part of DELL microcloud with 10Gbps NICs.
  • Node has 4xIntel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 32GB RAM.
• Network infrastructure:
  • 10GE network built by one Dell PowerConnect 8132 10GE switch (PCT8132).

Storage network performance

• Centos/MOS-6.0 — 9.4 Gbit/s
• Ubuntu/MOS-6.0 — 8.3 Gbit/s
• Ubuntu/MOS-6.1 — 9.4 Gbit/s

• CentOS/MOS-6.0 — 9.9 Gbit/s
• Ubuntu/MOS-6.0 — 9.4 Gbit/s
• Ubuntu/MOS-6.1 — 9.9 Gbit/s

Virtual network (VM to VM) performance (VLAN segmentation)

• CentOS/MOS-6.0 — 2.8 Gbit/s
• Ubuntu/MOS-6.0 — 3.8 Gbit/s
• Ubuntu/MOS-6.1 — 3.3 Gbit/s

• CentOS/MOS-6.0 — 7.4 Gbit/s
• Ubuntu/MOS-6.0 — 9.9 Gbit/s
• Ubuntu/MOS-6.1 — 9.9 Gbit/s

Virtual network (VM to VM) performance (GRE segmentation)

• Non-optimized network — 3.5 Gbit/s
• Optimized network — 9.7 Gbit/s