2012年10月18日木曜日

Quantum + OVS + GRE with Devstack (2)


このエントリーをはてなブックマークに追加


http://aikotobaha.blogspot.jp/2012/10/quantum-ovs-gre-with-devstack.html

の続き。

*10/19 図が間違っていたの修正


ルーティング

最初のネットワーク状態は以下


追加したネットワークに仮想マシンを追加してルーティングの状態を見てみる。

■add_net1へ追加
$ quantum net-list
+--------------------------------------+----------+--------------------------------------+
| id                                   | name     | subnets                              |
+--------------------------------------+----------+--------------------------------------+
| 2888da00-4060-4d2e-979b-d0f86390c76b | ext_net  | 9975adc4-8e2f-4152-a68c-b5b3137c0fab |
| b91768ec-84e2-4741-b254-a41f5bc43919 | add_net1 | 278814e8-0433-456c-ad8c-e62db9b0ebd0 |
| cb2f35e8-2ac7-493d-8089-a302e3be5cee | net1     | 7c10a3de-8e1f-4604-a243-ebedd51bf0ae |
| d560db49-30a4-46bc-a60e-9738017eac83 | add_net2 | f8be447d-e169-4970-8c33-44dcb1569689 |
+--------------------------------------+----------+--------------------------------------+

$ nova boot --flavor m1.tiny --image 69989707-d9ca-4905-aff7-39e3b025d704 --security-groups default --nic net-id=b91768ec-84e2-4741-b254-a41f5bc43919 testvm11
+------------------------+--------------------------------------+
| Property               | Value                                |
+------------------------+--------------------------------------+
| OS-DCF:diskConfig      | MANUAL                               |
| OS-EXT-STS:power_state | 0                                    |
| OS-EXT-STS:task_state  | scheduling                           |
| OS-EXT-STS:vm_state    | building                             |
| accessIPv4             |                                      |
| accessIPv6             |                                      |
| adminPass              | id5S62Ei5X8w                         |
| config_drive           |                                      |
| created                | 2012-10-17T13:56:22Z                 |
| flavor                 | m1.tiny                              |
| hostId                 |                                      |
| id                     | 0006e62a-2edf-46ad-b9ea-bd3bd6d1c3d2 |
| image                  | cirros-0.3.0-x86_64-uec              |
| key_name               | None                                 |
| metadata               | {}                                   |
| name                   | testvm11                             |
| progress               | 0                                    |
| security_groups        | [{u'name': u'default'}]              |
| status                 | BUILD                                |
| tenant_id              | 215c241925f543a1a69b3013b474fdd9     |
| updated                | 2012-10-17T13:56:22Z                 |
| user_id                | 9123dc08c8404ecd9cc5b5359bde48cc     |
+------------------------+--------------------------------------+

$ nova boot --flavor m1.tiny --image 69989707-d9ca-4905-aff7-39e3b025d704 --security-groups default --nic net-id=b91768ec-84e2-4741-b254-a41f5bc43919 testvm12
+------------------------+--------------------------------------+
| Property               | Value                                |
+------------------------+--------------------------------------+
| OS-DCF:diskConfig      | MANUAL                               |
| OS-EXT-STS:power_state | 0                                    |
| OS-EXT-STS:task_state  | scheduling                           |
| OS-EXT-STS:vm_state    | building                             |
| accessIPv4             |                                      |
| accessIPv6             |                                      |
| adminPass              | hE2BBb9kiYad                         |
| config_drive           |                                      |
| created                | 2012-10-17T13:57:43Z                 |
| flavor                 | m1.tiny                              |
| hostId                 |                                      |
| id                     | 0bb05517-b73c-4875-a630-ae6d88cb3837 |
| image                  | cirros-0.3.0-x86_64-uec              |
| key_name               | None                                 |
| metadata               | {}                                   |
| name                   | testvm12                             |
| progress               | 0                                    |
| security_groups        | [{u'name': u'default'}]              |
| status                 | BUILD                                |
| tenant_id              | 215c241925f543a1a69b3013b474fdd9     |
| updated                | 2012-10-17T13:57:44Z                 |
| user_id                | 9123dc08c8404ecd9cc5b5359bde48cc     |
+------------------------+--------------------------------------+

$ nova list
+--------------------------------------+----------+--------+---------------------+
| ID                                   | Name     | Status | Networks            |
+--------------------------------------+----------+--------+---------------------+
| 0006e62a-2edf-46ad-b9ea-bd3bd6d1c3d2 | testvm11 | ACTIVE | add_net1=172.50.0.2 |
| 0bb05517-b73c-4875-a630-ae6d88cb3837 | testvm12 | ACTIVE | add_net1=172.50.0.3 |
+--------------------------------------+----------+--------+---------------------+

OVSは以下の状態
devstack-cc$ sudo ovs-vsctl show
70d88f15-7f24-4fac-a509-3f3c30533cdb
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal 
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port "gre-2"
            Interface "gre-2"  
                type: gre
                options: {in_key=flow, out_key=flow, remote_ip="172.26.0.101"}
    Bridge br-int
        Port "tap43e85241-7d"  
            tag: 4
            Interface "tap43e85241-7d"
                type: internal 
        Port "qvoc5a0f828-8c"  
            tag: 3
            Interface "qvoc5a0f828-8c"
        Port "tapa9806132-f2"  
            tag: 1
            Interface "tapa9806132-f2"
                type: internal 
        Port "qr-01b9a7bf-ce"  
            tag: 1
            Interface "qr-01b9a7bf-ce"
                type: internal 
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tapd0eab10b-57"  
            tag: 3
            Interface "tapd0eab10b-57"
                type: internal 
        Port br-int
            Interface br-int   
                type: internal 
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-f616aed6-66"
            Interface "qg-f616aed6-66"
                type: internal
    ovs_version: "1.4.0+build0"

devstack-node$ sudo ovs-vsctl show
52a9d7d5-117d-49ed-8f79-ea4f43092994
    Bridge br-int
        Port "qvoa99c709b-91"
            tag: 2
            Interface "qvoa99c709b-91"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-1"
            Interface "gre-1"
                type: gre
                options: {in_key=flow, out_key=flow, remote_ip="172.26.0.100"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    ovs_version: "1.4.0+build0"

接続イメージは以下。



■仮想マシンのネットワーク接続
この2台の仮想マシンはmetadataサーバに接続できていない。

$ nova console-log --length 25 0006e62a-2edf-46ad-b9ea-bd3bd6d1c3d2
wget: can't connect to remote host (169.254.169.254): No route to host
wget: can't connect to remote host (169.254.169.254): No route to host
wget: can't connect to remote host (169.254.169.254): No route to host
  instance-id: 
  public-ipv4: 
  local-ipv4 : 
wget: can't connect to remote host (169.254.169.254): No route to host
cloud-userdata: failed to read instance id
WARN: /etc/rc3.d/S99-cloud-userdata failed
  ____               ____  ____
 / __/ __ ____ ____ / __ \/ __/
/ /__ / // __// __// /_/ /\ \ 
\___//_//_/  /_/   \____/___/ 
 http://launchpad.net/cirros


login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.
cirros login: 

$ nova console-log --length 25 0bb05517-b73c-4875-a630-ae6d88cb3837
wget: can't connect to remote host (169.254.169.254): No route to host
wget: can't connect to remote host (169.254.169.254): No route to host
wget: can't connect to remote host (169.254.169.254): No route to host
  instance-id: 
  public-ipv4: 
  local-ipv4 : 
wget: can't connect to remote host (169.254.169.254): No route to host
cloud-userdata: failed to read instance id
WARN: /etc/rc3.d/S99-cloud-userdata failed
  ____               ____  ____
 / __/ __ ____ ____ / __ \/ __/
/ /__ / // __// __// /_/ /\ \ 
\___//_//_/  /_/   \____/___/ 
 http://launchpad.net/cirros


login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.
cirros login: 

これはホスト側にGWとして設定した、172.50.0.254が存在していないため。

ただし仮想マシン同士の内部通信はできている。

$ sudo ip netns exec qdhcp-b91768ec-84e2-4741-b254-a41f5bc43919 ip addr list
21: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
23: tapd0eab10b-57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:78:21:ca brd ff:ff:ff:ff:ff:ff
    inet 172.50.0.1/24 brd 172.50.0.255 scope global tapd0eab10b-57
    inet6 fe80::f816:3eff:fe78:21ca/64 scope link 
       valid_lft forever preferred_lft forever

$ sudo ip netns exec qdhcp-b91768ec-84e2-4741-b254-a41f5bc43919 ip addr list
21: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
23: tapd0eab10b-57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:78:21:ca brd ff:ff:ff:ff:ff:ff
    inet 172.50.0.1/24 brd 172.50.0.255 scope global tapd0eab10b-57
    inet6 fe80::f816:3eff:fe78:21ca/64 scope link 
       valid_lft forever preferred_lft forever

$ sudo ip netns exec qdhcp-b91768ec-84e2-4741-b254-a41f5bc43919 ssh cirros@172.50.0.2
The authenticity of host '172.50.0.2 (172.50.0.2)' can't be established.
RSA key fingerprint is c7:ff:44:e0:1c:3b:6c:7c:6f:5b:e7:dc:e2:04:78:5c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.50.0.2' (RSA) to the list of known hosts.
cirros@172.50.0.2's password: 

$ ifconfig
eth0      Link encap:Ethernet  HWaddr FA:16:3E:49:68:D9  
          inet addr:172.50.0.2  Bcast:172.50.0.255  Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fe49:68d9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:802 errors:0 dropped:0 overruns:0 frame:0
          TX packets:530 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:130029 (126.9 KiB)  TX bytes:93945 (91.7 KiB)
          Interrupt:11 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:56 errors:0 dropped:0 overruns:0 frame:0
          TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:4928 (4.8 KiB)  TX bytes:4928 (4.8 KiB)

$ ping 172.50.0.3
PING 172.50.0.3 (172.50.0.3): 56 data bytes
64 bytes from 172.50.0.3: seq=0 ttl=64 time=430.830 ms
64 bytes from 172.50.0.3: seq=1 ttl=64 time=3.622 ms
64 bytes from 172.50.0.3: seq=2 ttl=64 time=1.766 ms
64 bytes from 172.50.0.3: seq=3 ttl=64 time=1.831 ms
^C
--- 172.50.0.3 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 1.766/109.512/430.830 ms

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.50.0.254    0.0.0.0         UG    0      0        0 eth0
172.50.0.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0
$ ping 172.50.0.254
PING 172.50.0.254 (172.50.0.254): 56 data bytes
^C
--- 172.50.0.254 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

■ゲートウェイの追加

$ quantum router-list
+--------------------------------------+---------+--------------------------------------------------------+
| id                                   | name    | external_gateway_info                                  |
+--------------------------------------+---------+--------------------------------------------------------+
| 34d4f254-4bfe-4f28-9ad5-e762e7014e6f | router1 | {"network_id": "2888da00-4060-4d2e-979b-d0f86390c76b"} |
+--------------------------------------+---------+--------------------------------------------------------+

$ quantum subnet-list
+--------------------------------------+------+----------------+--------------------------------------------------+
| id                                   | name | cidr           | allocation_pools                                 |
+--------------------------------------+------+----------------+--------------------------------------------------+
| 278814e8-0433-456c-ad8c-e62db9b0ebd0 |      | 172.50.0.0/24  | {"start": "172.50.0.1", "end": "172.50.0.253"}   |
| 7c10a3de-8e1f-4604-a243-ebedd51bf0ae |      | 172.24.17.0/24 | {"start": "172.24.17.1", "end": "172.24.17.253"} |
| f8be447d-e169-4970-8c33-44dcb1569689 |      | 172.100.0.0/24 | {"start": "172.100.0.1", "end": "172.100.0.253"} |
+--------------------------------------+------+----------------+--------------------------------------------------+

$ quantum router-interface-add 34d4f254-4bfe-4f28-9ad5-e762e7014e6f 278814e8-0433-456c-ad8c-e62db9b0ebd0
Added interface to router 34d4f254-4bfe-4f28-9ad5-e762e7014e6f

$ quantum port-list
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                            |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 01b9a7bf-ce0b-4146-b142-f2c99886de0b |      | fa:16:3e:54:79:55 | {"subnet_id": "7c10a3de-8e1f-4604-a243-ebedd51bf0ae", "ip_address": "172.24.17.254"} |
| 1f27a84a-85c6-4a18-867f-8e90bdb9eb0f |      | fa:16:3e:f0:9c:c5 | {"subnet_id": "278814e8-0433-456c-ad8c-e62db9b0ebd0", "ip_address": "172.50.0.254"}  |
| 43e85241-7d01-48ed-a577-99971d3c5637 |      | fa:16:3e:38:6b:bc | {"subnet_id": "f8be447d-e169-4970-8c33-44dcb1569689", "ip_address": "172.100.0.1"}   |
| a9806132-f2a7-4156-aec2-ed33264896c3 |      | fa:16:3e:56:12:af | {"subnet_id": "7c10a3de-8e1f-4604-a243-ebedd51bf0ae", "ip_address": "172.24.17.1"}   |
| a99c709b-91f3-47b9-adad-8e594c833eff |      | fa:16:3e:e0:17:b8 | {"subnet_id": "278814e8-0433-456c-ad8c-e62db9b0ebd0", "ip_address": "172.50.0.3"}    |
| c5a0f828-8c63-4389-8685-a792792147b0 |      | fa:16:3e:49:68:d9 | {"subnet_id": "278814e8-0433-456c-ad8c-e62db9b0ebd0", "ip_address": "172.50.0.2"}    |
| d0eab10b-5783-492a-9a61-9f123fd18bc9 |      | fa:16:3e:78:21:ca | {"subnet_id": "278814e8-0433-456c-ad8c-e62db9b0ebd0", "ip_address": "172.50.0.1"}    |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+

ゲートウェイを追加した状態のOVSとアドレスの状態
$ sudo ovs-vsctl show
70d88f15-7f24-4fac-a509-3f3c30533cdb
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port "gre-2"
            Interface "gre-2"
                type: gre
                options: {in_key=flow, out_key=flow, remote_ip="172.26.0.101"}
    Bridge br-int
        Port "tap43e85241-7d"
            tag: 4
            Interface "tap43e85241-7d"
                type: internal
        Port "qvoc5a0f828-8c"
            tag: 3
            Interface "qvoc5a0f828-8c"
        Port "tapa9806132-f2"
            tag: 1
            Interface "tapa9806132-f2"
                type: internal
        Port "qr-01b9a7bf-ce"
            tag: 1
            Interface "qr-01b9a7bf-ce"
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tapd0eab10b-57"
            tag: 3
            Interface "tapd0eab10b-57"
                type: internal
        Port "qr-1f27a84a-85"
            tag: 3
            Interface "qr-1f27a84a-85"
                type: internal
        Port br-int
            Interface br-int
                type: internal
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-f616aed6-66"
            Interface "qg-f616aed6-66"
                type: internal
    ovs_version: "1.4.0+build0"

$ ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:06:4e:63 brd ff:ff:ff:ff:ff:ff
    inet 192.168.128.100/24 brd 192.168.128.255 scope global eth0
    inet6 fe80::5054:ff:fe06:4e63/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:c5:1c:e5 brd ff:ff:ff:ff:ff:ff
    inet 172.26.0.100/24 brd 172.26.0.255 scope global eth1
    inet6 fe80::5054:ff:fec5:1ce5/64 scope link 
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:ba:e9:55 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:feba:e955/64 scope link 
       valid_lft forever preferred_lft forever
7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether 7a:03:9b:7a:a1:4e brd ff:ff:ff:ff:ff:ff
8: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:89:c7:7d:ed:4f brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 scope global br-ex
    inet6 fe80::5089:c7ff:fe7d:ed4f/64 scope link 
       valid_lft forever preferred_lft forever
10: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether 9a:7e:07:5d:f9:44 brd ff:ff:ff:ff:ff:ff
26: qbrc5a0f828-8c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether da:f4:6d:13:b9:9b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::98a9:3aff:fea7:2085/64 scope link 
       valid_lft forever preferred_lft forever
27: qvoc5a0f828-8c: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether ae:f0:ac:20:92:ec brd ff:ff:ff:ff:ff:ff
    inet6 fe80::acf0:acff:fe20:92ec/64 scope link 
       valid_lft forever preferred_lft forever
28: qvbc5a0f828-8c: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbrc5a0f828-8c state UP qlen 1000
    link/ether da:f4:6d:13:b9:9b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d8f4:6dff:fe13:b99b/64 scope link 
       valid_lft forever preferred_lft forever
29: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbrc5a0f828-8c state UNKNOWN qlen 500
    link/ether fe:16:3e:49:68:d9 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:fe49:68d9/64 scope link 
       valid_lft forever preferred_lft forever

$ for i in `sudo ip netns`; do echo; echo; echo ----- $i -----; sudo ip netns exec $i ip addr list; done
----- qdhcp-d560db49-30a4-46bc-a60e-9738017eac83 -----
24: tap43e85241-7d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:38:6b:bc brd ff:ff:ff:ff:ff:ff
    inet 172.100.0.1/24 brd 172.100.0.255 scope global tap43e85241-7d
    inet6 fe80::f816:3eff:fe38:6bbc/64 scope link 
       valid_lft forever preferred_lft forever
25: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

----- qdhcp-b91768ec-84e2-4741-b254-a41f5bc43919 -----
21: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
23: tapd0eab10b-57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:78:21:ca brd ff:ff:ff:ff:ff:ff
    inet 172.50.0.1/24 brd 172.50.0.255 scope global tapd0eab10b-57
    inet6 fe80::f816:3eff:fe78:21ca/64 scope link 
       valid_lft forever preferred_lft forever

----- qrouter-34d4f254-4bfe-4f28-9ad5-e762e7014e6f -----
13: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
14: qr-01b9a7bf-ce: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:54:79:55 brd ff:ff:ff:ff:ff:ff
    inet 172.24.17.254/24 brd 172.24.17.255 scope global qr-01b9a7bf-ce
    inet6 fe80::f816:3eff:fe54:7955/64 scope link 
       valid_lft forever preferred_lft forever
15: qg-f616aed6-66: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:ed:a6:2d brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global qg-f616aed6-66
    inet6 fe80::f816:3eff:feed:a62d/64 scope link 
       valid_lft forever preferred_lft forever
30: qr-1f27a84a-85: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:f0:9c:c5 brd ff:ff:ff:ff:ff:ff
    inet 172.50.0.254/24 brd 172.50.0.255 scope global qr-1f27a84a-85
    inet6 fe80::f816:3eff:fef0:9cc5/64 scope link 
       valid_lft forever preferred_lft forever

----- qdhcp-cb2f35e8-2ac7-493d-8089-a302e3be5cee -----
11: tapa9806132-f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:56:12:af brd ff:ff:ff:ff:ff:ff
    inet 172.24.17.1/24 brd 172.24.17.255 scope global tapa9806132-f2
    inet6 fe80::f816:3eff:fe56:12af/64 scope link 
       valid_lft forever preferred_lft forever
12: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever




この状態で、仮想マシンからは以下の通信はできる。
$ ifconfig -a
eth0      Link encap:Ethernet  HWaddr FA:16:3E:E0:17:B8  
          inet addr:172.50.0.3  Bcast:172.50.0.255  Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fee0:17b8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:879 errors:0 dropped:0 overruns:0 frame:0
          TX packets:613 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:146295 (142.8 KiB)  TX bytes:108251 (105.7 KiB)
          Interrupt:11 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:64 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5728 (5.5 KiB)  TX bytes:5728 (5.5 KiB)

$ ping 172.50.0.254
PING 172.50.0.254 (172.50.0.254): 56 data bytes
64 bytes from 172.50.0.254: seq=0 ttl=64 time=270.113 ms
64 bytes from 172.50.0.254: seq=1 ttl=64 time=7.303 ms

$ ping 172.24.17.254
PING 172.24.17.254 (172.24.17.254): 56 data bytes
64 bytes from 172.24.17.254: seq=0 ttl=64 time=4.946 ms
64 bytes from 172.24.17.254: seq=1 ttl=64 time=1.442 ms

$ ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: seq=0 ttl=63 time=326.001 ms
64 bytes from 10.0.0.1: seq=1 ttl=63 time=1.605 ms
64 bytes from 10.0.0.1: seq=2 ttl=63 time=1.614 ms

ここまでは到達できるが、これ以上は外に出られない。
metadata serverが起動している、192.168.128.100(169.254.169.254)には到達できない。

$ sudo ip netns exec qrouter-34d4f254-4bfe-4f28-9ad5-e762e7014e6f iptables -nvL -t nat
Chain quantum-l3-agent-PREROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    60 DNAT       tcp  --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 to:192.168.128.100:8775

これはルーティングテーブルの情報が不足しているため。

$ route -n
カーネルIP経路テーブル
受信先サイト    ゲートウェイ    ネットマスク   フラグ Metric Ref 使用数 インタフェース
0.0.0.0         192.168.128.1   0.0.0.0         UG    100    0        0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 br-ex
172.24.17.0     10.0.0.2        255.255.255.0   UG    0      0        0 br-ex
172.26.0.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.128.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

$ for i in `sudo ip netns`; do echo; echo; echo ----- $i -----; sudo ip netns exec $i route -n; done
----- qdhcp-d560db49-30a4-46bc-a60e-9738017eac83 -----
カーネルIP経路テーブル
受信先サイト    ゲートウェイ    ネットマスク   フラグ Metric Ref 使用数 インタフェース
172.100.0.0     0.0.0.0         255.255.255.0   U     0      0        0 tap43e85241-7d

----- qdhcp-b91768ec-84e2-4741-b254-a41f5bc43919 -----
カーネルIP経路テーブル
受信先サイト    ゲートウェイ    ネットマスク   フラグ Metric Ref 使用数 インタフェース
172.50.0.0      0.0.0.0         255.255.255.0   U     0      0        0 tapd0eab10b-57

----- qrouter-34d4f254-4bfe-4f28-9ad5-e762e7014e6f -----
カーネルIP経路テーブル
受信先サイト    ゲートウェイ    ネットマスク   フラグ Metric Ref 使用数 インタフェース
0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0 qg-f616aed6-66
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 qg-f616aed6-66
172.24.17.0     0.0.0.0         255.255.255.0   U     0      0        0 qr-01b9a7bf-ce
172.50.0.0      0.0.0.0         255.255.255.0   U     0      0        0 qr-1f27a84a-85

----- qdhcp-cb2f35e8-2ac7-493d-8089-a302e3be5cee -----
カーネルIP経路テーブル
受信先サイト    ゲートウェイ    ネットマスク   フラグ Metric Ref 使用数 インタフェース
172.24.17.0     0.0.0.0         255.255.255.0   U     0      0        0 tapa9806132-f2

手取り早く接続するには、ルーティングを追加してやる。
$ sudo route add -net 172.50.0.0/24 gw 10.0.0.2
$ route -n
カーネルIP経路テーブル
受信先サイト    ゲートウェイ    ネットマスク   フラグ Metric Ref 使用数 インタフェース
0.0.0.0         192.168.128.1   0.0.0.0         UG    100    0        0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 br-ex
172.24.17.0     10.0.0.2        255.255.255.0   UG    0      0        0 br-ex
172.26.0.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
172.50.0.0      10.0.0.2        255.255.255.0   UG    0      0        0 br-ex
192.168.128.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

これでmetadata server まで到達できる。
$ sudo ip netns exec qdhcp-b91768ec-84e2-4741-b254-a41f5bc43919 ssh cirros@172.50.0.2
cirros@172.50.0.2's password:

$ ping 192.168.128.100
PING 192.168.128.100 (192.168.128.100): 56 data bytes
64 bytes from 192.168.128.100: seq=0 ttl=63 time=17.921 ms
64 bytes from 192.168.128.100: seq=1 ttl=63 time=1.319 ms

$ wget http://169.254.169.254/2009-04-04/meta-data/instance-id
Connecting to 169.254.169.254 (169.254.169.254:80)
instance-id          100% |**********************|    10  --:--:-- ETA

$ cat instance-id
i-00000004

0 件のコメント:

コメントを投稿