1. 环境准备与基础配置在Ubuntu 24.04上部署OpenStack Yoga高可用集群首先需要做好基础环境准备。我建议使用至少三台物理服务器或虚拟机分别作为控制节点、计算节点和网络节点。实际项目中我曾遇到过单节点部署导致服务不可用的情况所以高可用架构真的非常重要。所有节点都需要配置静态IP地址建议使用192.168.137.60-62这样的连续地址段。修改/etc/hosts文件时记得所有节点要保持一致192.168.137.60 controller 192.168.137.61 compute1 192.168.137.62 compute2时间同步是很多新手容易忽略的问题。在分布式系统中时间不同步会导致各种诡异的问题。我习惯使用chrony进行时间同步sudo apt install -y chrony sudo systemctl enable --now chrony chronyc sources # 验证时间同步状态关闭防火墙可以避免很多连接问题但在生产环境中建议配置精确的防火墙规则sudo systemctl disable --now ufw添加OpenStack Yoga软件源是所有节点都需要进行的操作sudo add-apt-repository cloud-archive:yoga sudo apt update2. 数据库与消息队列配置高可用集群的核心是数据库和消息队列的可靠性。我推荐使用MariaDB Galera Cluster实现数据库高可用但在初期部署时可以先用单节点MariaDB简化配置。安装MariaDB时这些配置参数对性能影响很大sudo apt -y install mariadb-server python3-pymysql cat /etc/mysql/mariadb.conf.d/99-openstack.cnf EOF [mysqld] bind-address 192.168.137.60 default-storage-engine innodb innodb_file_per_table on max_connections 4096 collation-server utf8_general_ci character-set-server utf8 EOF sudo systemctl restart mysql sudo mysql_secure_installationRabbitMQ是OpenStack各组件通信的枢纽。我曾遇到过因为RabbitMQ配置不当导致消息堆积的问题所以这些参数要特别注意sudo apt -y install rabbitmq-server sudo rabbitmqctl add_user openstack 123456 sudo rabbitmqctl set_permissions openstack .* .* .* sudo rabbitmqctl set_policy HA-all {ha-mode:all,ha-sync-mode:automatic}Memcached用于缓存会话数据能显著提升性能sudo apt -y install memcached python3-memcache sudo sed -i s/-l 127.0.0.1/-l 192.168.137.60/ /etc/memcached.conf sudo systemctl restart memcached3. Keystone身份服务部署Keystone是OpenStack的身份认证核心它的高可用性直接影响整个平台。部署时我建议先创建数据库mysql -u root -p -e CREATE DATABASE keystone; GRANT ALL PRIVILEGES ON keystone.* TO keystonelocalhost IDENTIFIED BY keystone; GRANT ALL PRIVILEGES ON keystone.* TO keystone% IDENTIFIED BY keystone; 安装Keystone软件包时这些依赖项很重要sudo apt -y install keystone配置Keystone时Fernet令牌提供器是推荐选择比UUID更安全sudo cp /etc/keystone/keystone.conf{,.bak} sudo tee /etc/keystone/keystone.conf /dev/null EOF [database] connection mysqlpymysql://keystone:keystonecontroller/keystone [token] provider fernet EOF初始化数据库和Fernet密钥库sudo su -s /bin/sh -c keystone-manage db_sync keystone sudo keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone sudo keystone-manage credential_setup --keystone-user keystone --keystone-group keystone引导Keystone服务时这些URL参数要确保正确sudo keystone-manage bootstrap --bootstrap-password admin \ --bootstrap-admin-url http://controller:5000/v3/ \ --bootstrap-internal-url http://controller:5000/v3/ \ --bootstrap-public-url http://controller:5000/v3/ \ --bootstrap-region-id RegionOne配置Apache服务时ServerName设置经常被忽略echo ServerName controller | sudo tee -a /etc/apache2/apache2.conf sudo systemctl restart apache2创建管理员环境变量文件方便后续操作cat ~/admin-openrc EOF export OS_PROJECT_DOMAIN_NAMEDefault export OS_USER_DOMAIN_NAMEDefault export OS_PROJECT_NAMEadmin export OS_USERNAMEadmin export OS_PASSWORDadmin export OS_AUTH_URLhttp://controller:5000/v3 export OS_IDENTITY_API_VERSION3 export OS_IMAGE_API_VERSION2 EOF4. Glance镜像服务部署Glance管理虚拟机镜像它的高可用通过多副本存储实现。首先创建数据库mysql -u root -p -e CREATE DATABASE glance; GRANT ALL PRIVILEGES ON glance.* TO glancelocalhost IDENTIFIED BY glance; GRANT ALL PRIVILEGES ON glance.* TO glance% IDENTIFIED BY glance; 创建Glance服务凭证时这些角色绑定很关键source ~/admin-openrc openstack user create --domain default --password glance glance openstack role add --project service --user glance admin openstack service create --name glance --description OpenStack Image image配置Glance服务端点时三种类型的URL都要设置openstack endpoint create --region RegionOne \ image public http://controller:9292 openstack endpoint create --region RegionOne \ image internal http://controller:9292 openstack endpoint create --region RegionOne \ image admin http://controller:9292安装Glance软件包时注意这些依赖项sudo apt -y install glanceGlance的配置文件需要仔细调整特别是存储后端设置sudo cp /etc/glance/glance-api.conf{,.bak} sudo tee /etc/glance/glance-api.conf /dev/null EOF [DEFAULT] use_keystone_quotas True [database] connection mysqlpymysql://glance:glancecontroller/glance [keystone_authtoken] www_authenticate_uri http://controller:5000 auth_url http://controller:5000 memcached_servers controller:11211 auth_type password project_domain_name Default user_domain_name Default project_name service username glance password glance [paste_deploy] flavor keystone [glance_store] stores file,http default_store file filesystem_store_datadir /var/lib/glance/images/ EOF同步数据库并启动服务sudo su -s /bin/sh -c glance-manage db_sync glance sudo systemctl restart glance-api验证Glance服务是否正常openstack image list5. Nova计算服务部署Nova是OpenStack的核心计算服务它的高可用通过多计算节点实现。首先创建必要的数据库mysql -u root -p -e CREATE DATABASE nova; CREATE DATABASE nova_api; CREATE DATABASE nova_cell0; GRANT ALL PRIVILEGES ON nova.* TO novalocalhost IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova.* TO nova% IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_api.* TO novalocalhost IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_api.* TO nova% IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_cell0.* TO novalocalhost IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_cell0.* TO nova% IDENTIFIED BY nova; 创建Nova服务凭证source ~/admin-openrc openstack user create --domain default --password nova nova openstack role add --project service --user nova admin openstack service create --name nova --description OpenStack Compute compute配置Nova服务端点openstack endpoint create --region RegionOne \ compute public http://controller:8774/v2.1 openstack endpoint create --region RegionOne \ compute internal http://controller:8774/v2.1 openstack endpoint create --region RegionOne \ compute admin http://controller:8774/v2.1安装Nova控制节点组件sudo apt -y install nova-api nova-conductor nova-novncproxy nova-schedulerNova的配置文件较为复杂需要特别注意这些参数sudo cp /etc/nova/nova.conf{,.bak} sudo tee /etc/nova/nova.conf /dev/null EOF [DEFAULT] my_ip 192.168.137.60 transport_url rabbit://openstack:123456controller:5672/ use_neutron true firewall_driver nova.virt.firewall.NoopFirewallDriver [api] auth_strategy keystone [api_database] connection mysqlpymysql://nova:novacontroller/nova_api [database] connection mysqlpymysql://nova:novacontroller/nova [glance] api_servers http://controller:9292 [keystone_authtoken] www_authenticate_uri http://controller:5000/ auth_url http://controller:5000/ memcached_servers controller:11211 auth_type password project_domain_name Default user_domain_name Default project_name service username nova password nova [neutron] auth_url http://controller:5000 auth_type password project_domain_name Default user_domain_name Default region_name RegionOne project_name service username neutron password neutron [oslo_concurrency] lock_path /var/lib/nova/tmp [placement] region_name RegionOne project_domain_name Default project_name service auth_type password user_domain_name Default auth_url http://controller:5000/v3 username placement password placement [scheduler] discover_hosts_in_cells_interval 300 [vnc] enabled true server_listen \$my_ip server_proxyclient_address \$my_ip EOF同步Nova数据库并初始化cellsudo su -s /bin/sh -c nova-manage api_db sync nova sudo su -s /bin/sh -c nova-manage cell_v2 map_cell0 nova sudo su -s /bin/sh -c nova-manage cell_v2 create_cell --namecell1 --verbose nova sudo su -s /bin/sh -c nova-manage db sync nova验证cell是否创建成功sudo nova-manage cell_v2 list_cells启动Nova服务sudo systemctl enable --now nova-api nova-scheduler nova-conductor nova-novncproxy6. Neutron网络服务部署Neutron提供网络虚拟化功能它的高可用通过多网络节点实现。首先创建数据库mysql -u root -p -e CREATE DATABASE neutron; GRANT ALL PRIVILEGES ON neutron.* TO neutronlocalhost IDENTIFIED BY neutron; GRANT ALL PRIVILEGES ON neutron.* TO neutron% IDENTIFIED BY neutron; 创建Neutron服务凭证source ~/admin-openrc openstack user create --domain default --password neutron neutron openstack role add --project service --user neutron admin openstack service create --name neutron --description OpenStack Networking network配置Neutron服务端点openstack endpoint create --region RegionOne \ network public http://controller:9696 openstack endpoint create --region RegionOne \ network internal http://controller:9696 openstack endpoint create --region RegionOne \ network admin http://controller:9696安装Neutron服务组件sudo apt -y install neutron-server neutron-plugin-ml2 \ neutron-linuxbridge-agent neutron-l3-agent neutron-dhcp-agent \ neutron-metadata-agent配置Neutron主配置文件sudo cp /etc/neutron/neutron.conf{,.bak} sudo tee /etc/neutron/neutron.conf /dev/null EOF [DEFAULT] core_plugin ml2 service_plugins router allow_overlapping_ips true transport_url rabbit://openstack:123456controller:5672/ auth_strategy keystone notify_nova_on_port_status_changes true notify_nova_on_port_data_changes true [database] connection mysqlpymysql://neutron:neutroncontroller/neutron [keystone_authtoken] www_authenticate_uri http://controller:5000 auth_url http://controller:5000 memcached_servers controller:11211 auth_type password project_domain_name Default user_domain_name Default project_name service username neutron password neutron [nova] auth_url http://controller:5000 auth_type password project_domain_name Default user_domain_name Default region_name RegionOne project_name service username nova password nova [oslo_concurrency] lock_path /var/lib/neutron/tmp EOF配置ML2插件sudo cp /etc/neutron/plugins/ml2/ml2_conf.ini{,.bak} sudo tee /etc/neutron/plugins/ml2/ml2_conf.ini /dev/null EOF [ml2] type_drivers flat,vlan,vxlan tenant_network_types vxlan mechanism_drivers linuxbridge,l2population extension_drivers port_security [ml2_type_flat] flat_networks provider [ml2_type_vxlan] vni_ranges 1:1000 [securitygroup] enable_ipset true EOF配置Linux网桥代理sudo cp /etc/neutron/plugins/ml2/linuxbridge_agent.ini{,.bak} sudo tee /etc/neutron/plugins/ml2/linuxbridge_agent.ini /dev/null EOF [linux_bridge] physical_interface_mappings provider:ens33 [vxlan] enable_vxlan true local_ip 192.168.137.60 l2_population true [securitygroup] enable_security_group true firewall_driver neutron.agent.linux.iptables_firewall.IptablesFirewallDriver EOF配置L3代理sudo cp /etc/neutron/l3_agent.ini{,.bak} sudo tee /etc/neutron/l3_agent.ini /dev/null EOF [DEFAULT] interface_driver linuxbridge EOF配置DHCP代理sudo cp /etc/neutron/dhcp_agent.ini{,.bak} sudo tee /etc/neutron/dhcp_agent.ini /dev/null EOF [DEFAULT] interface_driver linuxbridge dhcp_driver neutron.agent.linux.dhcp.Dnsmasq enable_isolated_metadata true EOF配置元数据代理sudo cp /etc/neutron/metadata_agent.ini{,.bak} sudo tee /etc/neutron/metadata_agent.ini /dev/null EOF [DEFAULT] nova_metadata_host controller metadata_proxy_shared_secret metadata EOF同步Neutron数据库sudo su -s /bin/sh -c neutron-db-manage --config-file /etc/neutron/neutron.conf \ --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head neutron启动Neutron服务sudo systemctl enable --now neutron-server neutron-linuxbridge-agent \ neutron-dhcp-agent neutron-metadata-agent neutron-l3-agent7. Horizon仪表板部署Horizon提供Web管理界面虽然它本身无状态但可以通过负载均衡实现高可用。安装Horizon软件包sudo apt -y install openstack-dashboard配置Horizon时这些参数很关键sudo cp /etc/openstack-dashboard/local_settings.py{,.bak} sudo tee /etc/openstack-dashboard/local_settings.py /dev/null EOF OPENSTACK_HOST controller ALLOWED_HOSTS [*] SESSION_ENGINE django.contrib.sessions.backends.cache CACHES { default: { BACKEND: django.core.cache.backends.memcached.MemcachedCache, LOCATION: controller:11211, } } OPENSTACK_KEYSTONE_URL http://%s:5000/v3 % OPENSTACK_HOST OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT True OPENSTACK_API_VERSIONS { identity: 3, image: 2, volume: 3, } OPENSTACK_KEYSTONE_DEFAULT_DOMAIN Default OPENSTACK_KEYSTONE_DEFAULT_ROLE user TIME_ZONE Asia/Shanghai EOF重启Apache服务使配置生效sudo systemctl reload apache2.service8. Skyline智能运维集成Skyline为OpenStack提供智能监控和运维功能。首先创建数据库mysql -u root -p -e CREATE DATABASE skyline; GRANT ALL PRIVILEGES ON skyline.* TO skylinelocalhost IDENTIFIED BY skyline; GRANT ALL PRIVILEGES ON skyline.* TO skyline% IDENTIFIED BY skyline; 创建Skyline服务凭证source ~/admin-openrc openstack user create --domain default --password skyline skyline openstack role add --project service --user skyline admin创建Skyline配置文件sudo mkdir -p /etc/skyline sudo tee /etc/skyline/skyline.yaml /dev/null EOF default: database_url: mysql://skyline:skylinecontroller:3306/skyline debug: true log_dir: /var/log/skyline openstack: keystone_url: http://controller:5000/v3/ system_user_password: skyline EOF安装Docker并配置镜像加速sudo apt -y install docker.io sudo tee /etc/docker/daemon.json /dev/null EOF { registry-mirrors: [ https://hub-mirror.c.163.com, https://mirror.baidubce.com ] } EOF sudo systemctl restart docker运行Skyline容器sudo docker run -d --name skyline_bootstrap \ -e KOLLA_BOOTSTRAP \ -v /etc/skyline/skyline.yaml:/etc/skyline/skyline.yaml \ --nethost \ 99cloud/skyline:latest # 等待容器完成初始化 sudo docker logs -f skyline_bootstrap # 初始化完成后移除临时容器 sudo docker rm -f skyline_bootstrap # 启动正式Skyline容器 sudo docker run -d --name skyline \ -v /etc/skyline/skyline.yaml:/etc/skyline/skyline.yaml \ --nethost \ 99cloud/skyline:latest验证Skyline服务是否正常运行curl http://controller:80809. 高可用验证与故障测试完成所有组件部署后必须验证高可用性。我通常会进行这些测试停止控制节点上的MySQL服务验证其他节点是否仍能访问数据库断开一个计算节点的网络验证虚拟机是否自动迁移停止Neutron服务验证网络功能是否受影响模拟硬件故障验证服务自动恢复能力检查各服务状态openstack compute service list openstack network agent list openstack volume service list测试Skyline监控功能是否正常工作# 在Skyline界面查看各项指标是否正常采集 # 模拟服务故障验证告警是否触发10. 运维建议与经验分享在实际运维中我总结了这些经验定期备份数据库和配置文件我遇到过配置丢失导致服务不可用的情况监控关键指标API响应时间、消息队列积压、数据库连接数使用Skyline的预测功能提前发现潜在问题保持系统补丁更新但生产环境升级前一定要在测试环境验证文档化所有变更便于故障排查和团队协作对于性能调优这些参数很关键# Nova调度器配置 [scheduler] max_attempts 10 discover_hosts_in_cells_interval 300 # Neutron配置 [agent] report_interval 30 # RabbitMQ配置 queue_consumer_workers 4遇到问题时这些日志文件最有用/var/log/nova/nova-api.log /var/log/neutron/server.log /var/log/keystone/keystone.log /var/log/apache2/error.log docker logs skyline
Ubuntu 24.04上构建OpenStack Yoga高可用集群，并集成Skyline实现智能运维
1. 环境准备与基础配置在Ubuntu 24.04上部署OpenStack Yoga高可用集群首先需要做好基础环境准备。我建议使用至少三台物理服务器或虚拟机分别作为控制节点、计算节点和网络节点。实际项目中我曾遇到过单节点部署导致服务不可用的情况所以高可用架构真的非常重要。所有节点都需要配置静态IP地址建议使用192.168.137.60-62这样的连续地址段。修改/etc/hosts文件时记得所有节点要保持一致192.168.137.60 controller 192.168.137.61 compute1 192.168.137.62 compute2时间同步是很多新手容易忽略的问题。在分布式系统中时间不同步会导致各种诡异的问题。我习惯使用chrony进行时间同步sudo apt install -y chrony sudo systemctl enable --now chrony chronyc sources # 验证时间同步状态关闭防火墙可以避免很多连接问题但在生产环境中建议配置精确的防火墙规则sudo systemctl disable --now ufw添加OpenStack Yoga软件源是所有节点都需要进行的操作sudo add-apt-repository cloud-archive:yoga sudo apt update2. 数据库与消息队列配置高可用集群的核心是数据库和消息队列的可靠性。我推荐使用MariaDB Galera Cluster实现数据库高可用但在初期部署时可以先用单节点MariaDB简化配置。安装MariaDB时这些配置参数对性能影响很大sudo apt -y install mariadb-server python3-pymysql cat /etc/mysql/mariadb.conf.d/99-openstack.cnf EOF [mysqld] bind-address 192.168.137.60 default-storage-engine innodb innodb_file_per_table on max_connections 4096 collation-server utf8_general_ci character-set-server utf8 EOF sudo systemctl restart mysql sudo mysql_secure_installationRabbitMQ是OpenStack各组件通信的枢纽。我曾遇到过因为RabbitMQ配置不当导致消息堆积的问题所以这些参数要特别注意sudo apt -y install rabbitmq-server sudo rabbitmqctl add_user openstack 123456 sudo rabbitmqctl set_permissions openstack .* .* .* sudo rabbitmqctl set_policy HA-all {ha-mode:all,ha-sync-mode:automatic}Memcached用于缓存会话数据能显著提升性能sudo apt -y install memcached python3-memcache sudo sed -i s/-l 127.0.0.1/-l 192.168.137.60/ /etc/memcached.conf sudo systemctl restart memcached3. Keystone身份服务部署Keystone是OpenStack的身份认证核心它的高可用性直接影响整个平台。部署时我建议先创建数据库mysql -u root -p -e CREATE DATABASE keystone; GRANT ALL PRIVILEGES ON keystone.* TO keystonelocalhost IDENTIFIED BY keystone; GRANT ALL PRIVILEGES ON keystone.* TO keystone% IDENTIFIED BY keystone; 安装Keystone软件包时这些依赖项很重要sudo apt -y install keystone配置Keystone时Fernet令牌提供器是推荐选择比UUID更安全sudo cp /etc/keystone/keystone.conf{,.bak} sudo tee /etc/keystone/keystone.conf /dev/null EOF [database] connection mysqlpymysql://keystone:keystonecontroller/keystone [token] provider fernet EOF初始化数据库和Fernet密钥库sudo su -s /bin/sh -c keystone-manage db_sync keystone sudo keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone sudo keystone-manage credential_setup --keystone-user keystone --keystone-group keystone引导Keystone服务时这些URL参数要确保正确sudo keystone-manage bootstrap --bootstrap-password admin \ --bootstrap-admin-url http://controller:5000/v3/ \ --bootstrap-internal-url http://controller:5000/v3/ \ --bootstrap-public-url http://controller:5000/v3/ \ --bootstrap-region-id RegionOne配置Apache服务时ServerName设置经常被忽略echo ServerName controller | sudo tee -a /etc/apache2/apache2.conf sudo systemctl restart apache2创建管理员环境变量文件方便后续操作cat ~/admin-openrc EOF export OS_PROJECT_DOMAIN_NAMEDefault export OS_USER_DOMAIN_NAMEDefault export OS_PROJECT_NAMEadmin export OS_USERNAMEadmin export OS_PASSWORDadmin export OS_AUTH_URLhttp://controller:5000/v3 export OS_IDENTITY_API_VERSION3 export OS_IMAGE_API_VERSION2 EOF4. Glance镜像服务部署Glance管理虚拟机镜像它的高可用通过多副本存储实现。首先创建数据库mysql -u root -p -e CREATE DATABASE glance; GRANT ALL PRIVILEGES ON glance.* TO glancelocalhost IDENTIFIED BY glance; GRANT ALL PRIVILEGES ON glance.* TO glance% IDENTIFIED BY glance; 创建Glance服务凭证时这些角色绑定很关键source ~/admin-openrc openstack user create --domain default --password glance glance openstack role add --project service --user glance admin openstack service create --name glance --description OpenStack Image image配置Glance服务端点时三种类型的URL都要设置openstack endpoint create --region RegionOne \ image public http://controller:9292 openstack endpoint create --region RegionOne \ image internal http://controller:9292 openstack endpoint create --region RegionOne \ image admin http://controller:9292安装Glance软件包时注意这些依赖项sudo apt -y install glanceGlance的配置文件需要仔细调整特别是存储后端设置sudo cp /etc/glance/glance-api.conf{,.bak} sudo tee /etc/glance/glance-api.conf /dev/null EOF [DEFAULT] use_keystone_quotas True [database] connection mysqlpymysql://glance:glancecontroller/glance [keystone_authtoken] www_authenticate_uri http://controller:5000 auth_url http://controller:5000 memcached_servers controller:11211 auth_type password project_domain_name Default user_domain_name Default project_name service username glance password glance [paste_deploy] flavor keystone [glance_store] stores file,http default_store file filesystem_store_datadir /var/lib/glance/images/ EOF同步数据库并启动服务sudo su -s /bin/sh -c glance-manage db_sync glance sudo systemctl restart glance-api验证Glance服务是否正常openstack image list5. Nova计算服务部署Nova是OpenStack的核心计算服务它的高可用通过多计算节点实现。首先创建必要的数据库mysql -u root -p -e CREATE DATABASE nova; CREATE DATABASE nova_api; CREATE DATABASE nova_cell0; GRANT ALL PRIVILEGES ON nova.* TO novalocalhost IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova.* TO nova% IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_api.* TO novalocalhost IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_api.* TO nova% IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_cell0.* TO novalocalhost IDENTIFIED BY nova; GRANT ALL PRIVILEGES ON nova_cell0.* TO nova% IDENTIFIED BY nova; 创建Nova服务凭证source ~/admin-openrc openstack user create --domain default --password nova nova openstack role add --project service --user nova admin openstack service create --name nova --description OpenStack Compute compute配置Nova服务端点openstack endpoint create --region RegionOne \ compute public http://controller:8774/v2.1 openstack endpoint create --region RegionOne \ compute internal http://controller:8774/v2.1 openstack endpoint create --region RegionOne \ compute admin http://controller:8774/v2.1安装Nova控制节点组件sudo apt -y install nova-api nova-conductor nova-novncproxy nova-schedulerNova的配置文件较为复杂需要特别注意这些参数sudo cp /etc/nova/nova.conf{,.bak} sudo tee /etc/nova/nova.conf /dev/null EOF [DEFAULT] my_ip 192.168.137.60 transport_url rabbit://openstack:123456controller:5672/ use_neutron true firewall_driver nova.virt.firewall.NoopFirewallDriver [api] auth_strategy keystone [api_database] connection mysqlpymysql://nova:novacontroller/nova_api [database] connection mysqlpymysql://nova:novacontroller/nova [glance] api_servers http://controller:9292 [keystone_authtoken] www_authenticate_uri http://controller:5000/ auth_url http://controller:5000/ memcached_servers controller:11211 auth_type password project_domain_name Default user_domain_name Default project_name service username nova password nova [neutron] auth_url http://controller:5000 auth_type password project_domain_name Default user_domain_name Default region_name RegionOne project_name service username neutron password neutron [oslo_concurrency] lock_path /var/lib/nova/tmp [placement] region_name RegionOne project_domain_name Default project_name service auth_type password user_domain_name Default auth_url http://controller:5000/v3 username placement password placement [scheduler] discover_hosts_in_cells_interval 300 [vnc] enabled true server_listen \$my_ip server_proxyclient_address \$my_ip EOF同步Nova数据库并初始化cellsudo su -s /bin/sh -c nova-manage api_db sync nova sudo su -s /bin/sh -c nova-manage cell_v2 map_cell0 nova sudo su -s /bin/sh -c nova-manage cell_v2 create_cell --namecell1 --verbose nova sudo su -s /bin/sh -c nova-manage db sync nova验证cell是否创建成功sudo nova-manage cell_v2 list_cells启动Nova服务sudo systemctl enable --now nova-api nova-scheduler nova-conductor nova-novncproxy6. Neutron网络服务部署Neutron提供网络虚拟化功能它的高可用通过多网络节点实现。首先创建数据库mysql -u root -p -e CREATE DATABASE neutron; GRANT ALL PRIVILEGES ON neutron.* TO neutronlocalhost IDENTIFIED BY neutron; GRANT ALL PRIVILEGES ON neutron.* TO neutron% IDENTIFIED BY neutron; 创建Neutron服务凭证source ~/admin-openrc openstack user create --domain default --password neutron neutron openstack role add --project service --user neutron admin openstack service create --name neutron --description OpenStack Networking network配置Neutron服务端点openstack endpoint create --region RegionOne \ network public http://controller:9696 openstack endpoint create --region RegionOne \ network internal http://controller:9696 openstack endpoint create --region RegionOne \ network admin http://controller:9696安装Neutron服务组件sudo apt -y install neutron-server neutron-plugin-ml2 \ neutron-linuxbridge-agent neutron-l3-agent neutron-dhcp-agent \ neutron-metadata-agent配置Neutron主配置文件sudo cp /etc/neutron/neutron.conf{,.bak} sudo tee /etc/neutron/neutron.conf /dev/null EOF [DEFAULT] core_plugin ml2 service_plugins router allow_overlapping_ips true transport_url rabbit://openstack:123456controller:5672/ auth_strategy keystone notify_nova_on_port_status_changes true notify_nova_on_port_data_changes true [database] connection mysqlpymysql://neutron:neutroncontroller/neutron [keystone_authtoken] www_authenticate_uri http://controller:5000 auth_url http://controller:5000 memcached_servers controller:11211 auth_type password project_domain_name Default user_domain_name Default project_name service username neutron password neutron [nova] auth_url http://controller:5000 auth_type password project_domain_name Default user_domain_name Default region_name RegionOne project_name service username nova password nova [oslo_concurrency] lock_path /var/lib/neutron/tmp EOF配置ML2插件sudo cp /etc/neutron/plugins/ml2/ml2_conf.ini{,.bak} sudo tee /etc/neutron/plugins/ml2/ml2_conf.ini /dev/null EOF [ml2] type_drivers flat,vlan,vxlan tenant_network_types vxlan mechanism_drivers linuxbridge,l2population extension_drivers port_security [ml2_type_flat] flat_networks provider [ml2_type_vxlan] vni_ranges 1:1000 [securitygroup] enable_ipset true EOF配置Linux网桥代理sudo cp /etc/neutron/plugins/ml2/linuxbridge_agent.ini{,.bak} sudo tee /etc/neutron/plugins/ml2/linuxbridge_agent.ini /dev/null EOF [linux_bridge] physical_interface_mappings provider:ens33 [vxlan] enable_vxlan true local_ip 192.168.137.60 l2_population true [securitygroup] enable_security_group true firewall_driver neutron.agent.linux.iptables_firewall.IptablesFirewallDriver EOF配置L3代理sudo cp /etc/neutron/l3_agent.ini{,.bak} sudo tee /etc/neutron/l3_agent.ini /dev/null EOF [DEFAULT] interface_driver linuxbridge EOF配置DHCP代理sudo cp /etc/neutron/dhcp_agent.ini{,.bak} sudo tee /etc/neutron/dhcp_agent.ini /dev/null EOF [DEFAULT] interface_driver linuxbridge dhcp_driver neutron.agent.linux.dhcp.Dnsmasq enable_isolated_metadata true EOF配置元数据代理sudo cp /etc/neutron/metadata_agent.ini{,.bak} sudo tee /etc/neutron/metadata_agent.ini /dev/null EOF [DEFAULT] nova_metadata_host controller metadata_proxy_shared_secret metadata EOF同步Neutron数据库sudo su -s /bin/sh -c neutron-db-manage --config-file /etc/neutron/neutron.conf \ --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head neutron启动Neutron服务sudo systemctl enable --now neutron-server neutron-linuxbridge-agent \ neutron-dhcp-agent neutron-metadata-agent neutron-l3-agent7. Horizon仪表板部署Horizon提供Web管理界面虽然它本身无状态但可以通过负载均衡实现高可用。安装Horizon软件包sudo apt -y install openstack-dashboard配置Horizon时这些参数很关键sudo cp /etc/openstack-dashboard/local_settings.py{,.bak} sudo tee /etc/openstack-dashboard/local_settings.py /dev/null EOF OPENSTACK_HOST controller ALLOWED_HOSTS [*] SESSION_ENGINE django.contrib.sessions.backends.cache CACHES { default: { BACKEND: django.core.cache.backends.memcached.MemcachedCache, LOCATION: controller:11211, } } OPENSTACK_KEYSTONE_URL http://%s:5000/v3 % OPENSTACK_HOST OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT True OPENSTACK_API_VERSIONS { identity: 3, image: 2, volume: 3, } OPENSTACK_KEYSTONE_DEFAULT_DOMAIN Default OPENSTACK_KEYSTONE_DEFAULT_ROLE user TIME_ZONE Asia/Shanghai EOF重启Apache服务使配置生效sudo systemctl reload apache2.service8. Skyline智能运维集成Skyline为OpenStack提供智能监控和运维功能。首先创建数据库mysql -u root -p -e CREATE DATABASE skyline; GRANT ALL PRIVILEGES ON skyline.* TO skylinelocalhost IDENTIFIED BY skyline; GRANT ALL PRIVILEGES ON skyline.* TO skyline% IDENTIFIED BY skyline; 创建Skyline服务凭证source ~/admin-openrc openstack user create --domain default --password skyline skyline openstack role add --project service --user skyline admin创建Skyline配置文件sudo mkdir -p /etc/skyline sudo tee /etc/skyline/skyline.yaml /dev/null EOF default: database_url: mysql://skyline:skylinecontroller:3306/skyline debug: true log_dir: /var/log/skyline openstack: keystone_url: http://controller:5000/v3/ system_user_password: skyline EOF安装Docker并配置镜像加速sudo apt -y install docker.io sudo tee /etc/docker/daemon.json /dev/null EOF { registry-mirrors: [ https://hub-mirror.c.163.com, https://mirror.baidubce.com ] } EOF sudo systemctl restart docker运行Skyline容器sudo docker run -d --name skyline_bootstrap \ -e KOLLA_BOOTSTRAP \ -v /etc/skyline/skyline.yaml:/etc/skyline/skyline.yaml \ --nethost \ 99cloud/skyline:latest # 等待容器完成初始化 sudo docker logs -f skyline_bootstrap # 初始化完成后移除临时容器 sudo docker rm -f skyline_bootstrap # 启动正式Skyline容器 sudo docker run -d --name skyline \ -v /etc/skyline/skyline.yaml:/etc/skyline/skyline.yaml \ --nethost \ 99cloud/skyline:latest验证Skyline服务是否正常运行curl http://controller:80809. 高可用验证与故障测试完成所有组件部署后必须验证高可用性。我通常会进行这些测试停止控制节点上的MySQL服务验证其他节点是否仍能访问数据库断开一个计算节点的网络验证虚拟机是否自动迁移停止Neutron服务验证网络功能是否受影响模拟硬件故障验证服务自动恢复能力检查各服务状态openstack compute service list openstack network agent list openstack volume service list测试Skyline监控功能是否正常工作# 在Skyline界面查看各项指标是否正常采集 # 模拟服务故障验证告警是否触发10. 运维建议与经验分享在实际运维中我总结了这些经验定期备份数据库和配置文件我遇到过配置丢失导致服务不可用的情况监控关键指标API响应时间、消息队列积压、数据库连接数使用Skyline的预测功能提前发现潜在问题保持系统补丁更新但生产环境升级前一定要在测试环境验证文档化所有变更便于故障排查和团队协作对于性能调优这些参数很关键# Nova调度器配置 [scheduler] max_attempts 10 discover_hosts_in_cells_interval 300 # Neutron配置 [agent] report_interval 30 # RabbitMQ配置 queue_consumer_workers 4遇到问题时这些日志文件最有用/var/log/nova/nova-api.log /var/log/neutron/server.log /var/log/keystone/keystone.log /var/log/apache2/error.log docker logs skyline
