SETUP » История » Версия 15
Владимир Ипатов, 12.11.2012 02:59
1 | 1 | Dmitry Chernyak | h1. SETUP |
---|---|---|---|
2 | 1 | Dmitry Chernyak | |
3 | 1 | Dmitry Chernyak | {{toc}} |
4 | 1 | Dmitry Chernyak | |
5 | 1 | Dmitry Chernyak | Ensure both nodes are up. |
6 | 1 | Dmitry Chernyak | |
7 | 1 | Dmitry Chernyak | If you planning to use the secondary network for SAN and DRBD synchronization, you |
8 | 1 | Dmitry Chernyak | should configure secondary IP interfaces manually on both nodes at this time. |
9 | 1 | Dmitry Chernyak | |
10 | 11 | Dmitry Chernyak | Log in to the first node via ssh. Due to lack of DNS there may be a minute timeout |
11 | 11 | Dmitry Chernyak | before the server anwers yoy with the password prompt. |
12 | 1 | Dmitry Chernyak | |
13 | 2 | Владимир Ипатов | h2. NETWORK CONFIGURATION |
14 | 2 | Владимир Ипатов | |
15 | 2 | Владимир Ипатов | Network configuration may be highly various. |
16 | 11 | Dmitry Chernyak | Here we describe several usual schemas. |
17 | 2 | Владимир Ипатов | |
18 | 7 | Владимир Ипатов | h3. Basic schema - one ethernet to all. |
19 | 2 | Владимир Ипатов | |
20 | 11 | Dmitry Chernyak | One ethernet, one subnet, internet connection provided by external (not in claster) router. |
21 | 1 | Dmitry Chernyak | By default installer create bridge named xen-br0. You can customize parameters by editing /etc/network/interfaces. |
22 | 7 | Владимир Ипатов | In this case you must have nodes connected to gigabit ethernet switch. |
23 | 3 | Владимир Ипатов | By default it looks like: |
24 | 3 | Владимир Ипатов | <pre> |
25 | 3 | Владимир Ипатов | auto xen-br0 |
26 | 3 | Владимир Ипатов | iface xen-br0 inet static |
27 | 3 | Владимир Ипатов | address 192.168.5.88 |
28 | 3 | Владимир Ипатов | netmask 255.255.255.0 |
29 | 3 | Владимир Ипатов | network 192.168.5.0 |
30 | 3 | Владимир Ипатов | broadcast 192.168.5.255 |
31 | 3 | Владимир Ипатов | gateway 192.168.5.1 |
32 | 3 | Владимир Ипатов | bridge_ports eth0 |
33 | 3 | Владимир Ипатов | bridge_stp off |
34 | 3 | Владимир Ипатов | bridge_fd 0 |
35 | 1 | Dmitry Chernyak | # up ifconfig eth0 mtu 9000 |
36 | 1 | Dmitry Chernyak | # up ifconfig xen-br0 mtu 9000 |
37 | 3 | Владимир Ипатов | </pre> |
38 | 1 | Dmitry Chernyak | Important parameters besides ipv4 settings is: |
39 | 11 | Dmitry Chernyak | <pre> |
40 | 11 | Dmitry Chernyak | bridge_ports eth0 |
41 | 11 | Dmitry Chernyak | </pre> |
42 | 1 | Dmitry Chernyak | |
43 | 11 | Dmitry Chernyak | - means that physical interface eth0 enslaved to this bridge. |
44 | 11 | Dmitry Chernyak | |
45 | 11 | Dmitry Chernyak | <pre> |
46 | 1 | Dmitry Chernyak | up ifconfig eth0 mtu 9000 |
47 | 11 | Dmitry Chernyak | up ifconfig xen-br0 mtu 9000 |
48 | 11 | Dmitry Chernyak | </pre> |
49 | 11 | Dmitry Chernyak | |
50 | 11 | Dmitry Chernyak | - setting jumbo frame on bridge for more network speed and less cpu utilization. |
51 | 3 | Владимир Ипатов | It will be actual on interface where drbd link will be. |
52 | 3 | Владимир Ипатов | However, setting mtu higher than 1500 will cause problems with any network equipment that |
53 | 1 | Dmitry Chernyak | doesn't support jumbo frames. That's the reason because it option commented out by default. |
54 | 3 | Владимир Ипатов | |
55 | 7 | Владимир Ипатов | h3. Default schema - two ethernets, one for interlink(ganeti interoperation+drbd link) and one for lan. |
56 | 1 | Dmitry Chernyak | |
57 | 11 | Dmitry Chernyak | This schema suits most cases. It doesn't required a gigabit switch, provide good performance and reliability. |
58 | 15 | Владимир Ипатов | Two gigabit network interfaces on the nodes are connected directly or via a gigabit |
59 | 15 | Владимир Ипатов | switch (if you want more than two nodes in the cluster). |
60 | 14 | Владимир Ипатов | Other interfaces connected to lan. |
61 | 7 | Владимир Ипатов | Lan failure doesn't affect cluster in this setup. |
62 | 8 | Владимир Ипатов | This is /etc/network/interfaces file for this setup: |
63 | 8 | Владимир Ипатов | <pre>auto xen-br0 |
64 | 8 | Владимир Ипатов | iface xen-br0 inet static |
65 | 8 | Владимир Ипатов | address 192.168.236.1 |
66 | 8 | Владимир Ипатов | netmask 255.255.255.0 |
67 | 8 | Владимир Ипатов | network 192.168.236.0 |
68 | 8 | Владимир Ипатов | broadcast 192.168.236.255 |
69 | 8 | Владимир Ипатов | bridge_ports eth0 |
70 | 1 | Dmitry Chernyak | bridge_stp off |
71 | 1 | Dmitry Chernyak | bridge_fd 0 |
72 | 1 | Dmitry Chernyak | # up ifconfig eth0 mtu 9000 |
73 | 1 | Dmitry Chernyak | # up ifconfig xen-br0 mtu 9000 |
74 | 8 | Владимир Ипатов | |
75 | 7 | Владимир Ипатов | auto xen-lan |
76 | 8 | Владимир Ипатов | iface xen-lan inet static |
77 | 8 | Владимир Ипатов | address 192.168.5.55 |
78 | 8 | Владимир Ипатов | netmask 255.255.255.0 |
79 | 8 | Владимир Ипатов | network 192.168.5.0 |
80 | 1 | Dmitry Chernyak | broadcast 192.168.5.255 |
81 | 12 | Владимир Ипатов | gateway 192.168.5.1 |
82 | 8 | Владимир Ипатов | bridge_ports eth1 |
83 | 8 | Владимир Ипатов | bridge_stp off |
84 | 8 | Владимир Ипатов | bridge_fd 0 |
85 | 8 | Владимир Ипатов | </pre> |
86 | 11 | Dmitry Chernyak | |
87 | 11 | Dmitry Chernyak | xen-br0 used by ganeti interoperation and drbd link, it was configured by the installer. |
88 | 11 | Dmitry Chernyak | Also the dns server and the gateway was configured by the installer - it will be our service instance(sci) address. |
89 | 11 | Dmitry Chernyak | xen-lan used by lan connection, its configuration must be added by hands. |
90 | 8 | Владимир Ипатов | In this network configuration you must fill these variables in sci.conf: |
91 | 9 | Владимир Ипатов | NODE1_IP - already configured by installer. |
92 | 9 | Владимир Ипатов | NODE1_NAME - already configured by installer. |
93 | 9 | Владимир Ипатов | NODE2_IP - set interlink ip address of second node. e.g. 192.168.236.2 |
94 | 1 | Dmitry Chernyak | NODE2_NAME - set second node name. e.g. gnt2 |
95 | 9 | Владимир Ипатов | NODE1_LAN_IP - lan ip for first node. It will be available by dns name $NODE1_NAME-lan. 192.168.5.55 |
96 | 15 | Владимир Ипатов | NODE2_LAN_IP - lan ip for second node. It will be available by dns name $NODE2_NAME-lan. e.g. 192.168.5.58 |
97 | 1 | Dmitry Chernyak | CLUSTER_IP - cluster address in lan. Must not match any exist host address in lan. 192.168.5.35 |
98 | 9 | Владимир Ипатов | CLUSTER_NAME - cluster name in lan. In will be available by dns name $CLUSTER_NAME. |
99 | 15 | Владимир Ипатов | SCI_LAN_IP - if you want presence sci intance in your lan, assign ip. e.g. 192.168.5.59 |
100 | 7 | Владимир Ипатов | |
101 | 7 | Владимир Ипатов | h3. Mupltiple bridges with routing, firewalling and wan access. |
102 | 7 | Владимир Ипатов | |
103 | 3 | Владимир Ипатов | Here is a bit more complicated network setup. |
104 | 3 | Владимир Ипатов | In this setup we have, for example, two private netwokrs and wan by ethernet. All routing and firewalling |
105 | 7 | Владимир Ипатов | performed by separate firewall instance in our cluster. This setup fit when you don't have expensive hardware routers and firewalls. |
106 | 3 | Владимир Ипатов | This is /etc/network/interfaces file in this setup: |
107 | 3 | Владимир Ипатов | <pre> |
108 | 5 | Владимир Ипатов | auto lan |
109 | 5 | Владимир Ипатов | iface lan inet static |
110 | 5 | Владимир Ипатов | address 192.168.21.10 |
111 | 5 | Владимир Ипатов | netmask 255.255.255.0 |
112 | 1 | Dmitry Chernyak | bridge_ports eth0 |
113 | 1 | Dmitry Chernyak | bridge_stp off |
114 | 1 | Dmitry Chernyak | bridge_fd 0 |
115 | 5 | Владимир Ипатов | |
116 | 12 | Владимир Ипатов | auto server |
117 | 12 | Владимир Ипатов | iface server inet static |
118 | 5 | Владимир Ипатов | address 192.168.20.10 |
119 | 5 | Владимир Ипатов | netmask 255.255.255.0 |
120 | 5 | Владимир Ипатов | gateway 192.168.20.1 |
121 | 5 | Владимир Ипатов | bridge_ports eth1 |
122 | 5 | Владимир Ипатов | bridge_stp off |
123 | 5 | Владимир Ипатов | bridge_fd 0 |
124 | 5 | Владимир Ипатов | up ifconfig eth1 mtu 9000 |
125 | 5 | Владимир Ипатов | up ifconfig dmz mtu 9000 |
126 | 5 | Владимир Ипатов | |
127 | 5 | Владимир Ипатов | auto wan1 |
128 | 5 | Владимир Ипатов | iface wan1 inet manual |
129 | 5 | Владимир Ипатов | bridge_ports eth2 |
130 | 5 | Владимир Ипатов | bridge_stp off |
131 | 5 | Владимир Ипатов | bridge_fd 0 |
132 | 12 | Владимир Ипатов | </pre> |
133 | 12 | Владимир Ипатов | In this example we have separate lan interfaces, server interface(in this case servers separated from lan and |
134 | 5 | Владимир Ипатов | clients go to servers thru router) and wan interface. server interface - ganeti interoperation dev and drbd link |
135 | 6 | Владимир Ипатов | interfase, so there is mtu 9000. |
136 | 1 | Dmitry Chernyak | There is no address in wan for hypervisor, although we recommend you to get subnet from |
137 | 1 | Dmitry Chernyak | your ISP in order to assign IP addresses to nodes to management it even if router instance |
138 | 1 | Dmitry Chernyak | is down. |
139 | 15 | Владимир Ипатов | In this network configuration you must fill these variables in sci.conf: |
140 | 15 | Владимир Ипатов | NODE1_IP - already configured by installer. |
141 | 15 | Владимир Ипатов | NODE1_NAME - already configured by installer. |
142 | 15 | Владимир Ипатов | NODE2_IP - set interlink ip address of second node. e.g. 192.168.20.2 |
143 | 15 | Владимир Ипатов | NODE2_NAME - set second node name. e.g. gnt2 |
144 | 15 | Владимир Ипатов | NODE1_LAN_IP - lan ip for first node. It will be available by dns name $NODE1_NAME-lan. 192.168.21.55 |
145 | 15 | Владимир Ипатов | NODE2_LAN_IP - lan ip for second node. It will be available by dns name $NODE2_NAME-lan. e.g. 192.168.21.58 |
146 | 15 | Владимир Ипатов | CLUSTER_IP - cluster address in lan. Must not match any exist host address in lan. 192.168.21.35 |
147 | 15 | Владимир Ипатов | CLUSTER_NAME - cluster name in lan. In will be available by dns name $CLUSTER_NAME. |
148 | 15 | Владимир Ипатов | SCI_LAN_IP - if you want presence sci intance in your lan, assign ip. e.g. 192.168.21.59 |
149 | 5 | Владимир Ипатов | |
150 | 5 | Владимир Ипатов | Here is an example /etc/network/interfaces in router instance: |
151 | 5 | Владимир Ипатов | <pre> |
152 | 5 | Владимир Ипатов | auto eth0 |
153 | 5 | Владимир Ипатов | iface eth0 inet static |
154 | 5 | Владимир Ипатов | address 192.168.20.1 |
155 | 5 | Владимир Ипатов | netmask 255.255.255.0 |
156 | 5 | Владимир Ипатов | |
157 | 5 | Владимир Ипатов | auto eth1 |
158 | 5 | Владимир Ипатов | iface eth1 inet static |
159 | 5 | Владимир Ипатов | address 192.168.21.1 |
160 | 5 | Владимир Ипатов | netmask 255.255.255.0 |
161 | 5 | Владимир Ипатов | |
162 | 5 | Владимир Ипатов | auto eth2 |
163 | 5 | Владимир Ипатов | iface eth2 inet static |
164 | 1 | Dmitry Chernyak | address 1.1.1.2 |
165 | 5 | Владимир Ипатов | netmask 255.255.255.0 |
166 | 1 | Dmitry Chernyak | address 1.1.1.1 |
167 | 1 | Dmitry Chernyak | </pre> |
168 | 15 | Владимир Ипатов | Where eth0 linked to bridge server, eth1 linked to lan, eth2 linked to wan. |
169 | 7 | Владимир Ипатов | |
170 | 7 | Владимир Ипатов | h3. Datacenter schema - separate interfaces for lan, ganeti interoperation, drbd link. |
171 | 10 | Владимир Ипатов | |
172 | 7 | Владимир Ипатов | If you have powerful networking infrastructure |
173 | 13 | Владимир Ипатов | Here we have separate interfaces for ganeti interoperation(in this case it may be named management interface) |
174 | 13 | Владимир Ипатов | <pre>auto mgmt |
175 | 13 | Владимир Ипатов | iface mgmt inet static |
176 | 13 | Владимир Ипатов | address 192.168.236.1 |
177 | 13 | Владимир Ипатов | netmask 255.255.255.0 |
178 | 13 | Владимир Ипатов | network 192.168.236.0 |
179 | 13 | Владимир Ипатов | gateway 192.168.236.1 |
180 | 13 | Владимир Ипатов | broadcast 192.168.236.255 |
181 | 13 | Владимир Ипатов | bridge_ports eth0 |
182 | 13 | Владимир Ипатов | bridge_stp off |
183 | 13 | Владимир Ипатов | bridge_fd 0 |
184 | 13 | Владимир Ипатов | |
185 | 13 | Владимир Ипатов | auto xen-san |
186 | 13 | Владимир Ипатов | iface xen-san inet static |
187 | 13 | Владимир Ипатов | address 192.168.237.1 |
188 | 13 | Владимир Ипатов | netmask 255.255.255.0 |
189 | 13 | Владимир Ипатов | network 192.168.237.0 |
190 | 13 | Владимир Ипатов | broadcast 192.168.237.255 |
191 | 13 | Владимир Ипатов | bridge_ports eth1 |
192 | 13 | Владимир Ипатов | bridge_stp off |
193 | 13 | Владимир Ипатов | bridge_fd 0 |
194 | 13 | Владимир Ипатов | up ifconfig eth1 mtu 9000 |
195 | 13 | Владимир Ипатов | up ifconfig xen-san mtu 9000 |
196 | 13 | Владимир Ипатов | |
197 | 13 | Владимир Ипатов | auto xen-lan |
198 | 13 | Владимир Ипатов | iface xen-lan inet manual |
199 | 13 | Владимир Ипатов | bridge_ports eth2 |
200 | 13 | Владимир Ипатов | bridge_stp off |
201 | 13 | Владимир Ипатов | bridge_fd 0 |
202 | 13 | Владимир Ипатов | </pre> |
203 | 13 | Владимир Ипатов | |
204 | 13 | Владимир Ипатов | In this example nodes don't have addresses in lan. |
205 | 13 | Владимир Ипатов | You must fill these vars in sci.conf to create cluster fits this network config: |
206 | 1 | Dmitry Chernyak | NODE1_IP - already configured by installer. |
207 | 13 | Владимир Ипатов | NODE1_NAME - already configured by installer. |
208 | 13 | Владимир Ипатов | NODE2_IP - set interlink ip address of second node. e.g. 192.168.236.2 |
209 | 13 | Владимир Ипатов | NODE2_NAME - set second node name. e.g. gnt2 |
210 | 13 | Владимир Ипатов | NODE1_SAN_IP - lan ip for first node. It will be available by dns name $NODE1_NAME-san. 192.168.237.1 |
211 | 15 | Владимир Ипатов | NODE2_SAN_IP - lan ip for second node. It will be available by dns name $NODE2_NAME-san. e.g. 192.168.237.2 |
212 | 13 | Владимир Ипатов | CLUSTER_IP - cluster address in lan. Must not match any exist host address in lan. 192.168.236.35 |
213 | 13 | Владимир Ипатов | CLUSTER_NAME - cluster name in lan. In will be available by dns name $CLUSTER_NAME. |
214 | 13 | Владимир Ипатов | SCI_LAN_IP - if you want presence sci intance in your lan, assign ip. e.g. 192.168.35.5 |
215 | 13 | Владимир Ипатов | SCI_LAN_NETMASK - your nodes don't have addresses in lan, so you must enter netmask for this segment by hand. e.g. 255.255.255.0 |
216 | 13 | Владимир Ипатов | SCI_LAN_GATEWAY - your nodes don't have addresses in lan, so you must enter gateway for this segment by hand. e.g. 192.168.35.1 |
217 | 13 | Владимир Ипатов | Of course, it is easy to use VLANS in datacenter conditions. Next example will explain how. However, remember it is recommended |
218 | 13 | Владимир Ипатов | that drbd link must be on separate ethernet. |
219 | 13 | Владимир Ипатов | |
220 | 5 | Владимир Ипатов | |
221 | 5 | Владимир Ипатов | h3. VLAN schema |
222 | 5 | Владимир Ипатов | |
223 | 5 | Владимир Ипатов | If you have managed switches, you can set networking with VLANs. |
224 | 5 | Владимир Ипатов | You should add something like this for each VLAN: |
225 | 5 | Владимир Ипатов | <pre> |
226 | 5 | Владимир Ипатов | auto eth0.55 |
227 | 5 | Владимир Ипатов | iface eth0.55 inet manual |
228 | 5 | Владимир Ипатов | up ifconfig eth0.55 up |
229 | 5 | Владимир Ипатов | |
230 | 5 | Владимир Ипатов | auto bridge-example-vlan |
231 | 5 | Владимир Ипатов | iface bridge-example-vlan inet manual |
232 | 5 | Владимир Ипатов | up brctl addbr bridge-example-vlan |
233 | 5 | Владимир Ипатов | up brctl addif bridge-example-vlan eth0.55 |
234 | 5 | Владимир Ипатов | up brctl stp bridge-example-vlan off |
235 | 5 | Владимир Ипатов | up ifconfig bridge-example-vlan up |
236 | 5 | Владимир Ипатов | down ifconfig bridge-example-vlan down |
237 | 5 | Владимир Ипатов | down brctl delbr bridge-example-vlan |
238 | 5 | Владимир Ипатов | </pre> |
239 | 5 | Владимир Ипатов | Where 55 - VLAN number. |
240 | 5 | Владимир Ипатов | In this example node don't have an ip address in this VLAN, although you could |
241 | 5 | Владимир Ипатов | assign an ip to bridge just like standard bridge. |
242 | 5 | Владимир Ипатов | |
243 | 5 | Владимир Ипатов | Alternative schema is: |
244 | 5 | Владимир Ипатов | <pre> |
245 | 5 | Владимир Ипатов | auto vlan55 |
246 | 5 | Владимир Ипатов | iface vlan55 inet manual |
247 | 5 | Владимир Ипатов | vlan_raw_device eth0 |
248 | 5 | Владимир Ипатов | |
249 | 5 | Владимир Ипатов | auto bridge-example-vlan |
250 | 5 | Владимир Ипатов | iface bridge-example-vlan inet manual |
251 | 5 | Владимир Ипатов | bridge_ports vlan55 |
252 | 5 | Владимир Ипатов | bridge_stp off |
253 | 5 | Владимир Ипатов | bridge_fd 0 |
254 | 2 | Владимир Ипатов | </pre> |
255 | 1 | Dmitry Chernyak | It do the same, but in another way. |
256 | 1 | Dmitry Chernyak | |
257 | 1 | Dmitry Chernyak | h2. DEFINING ENVIRONMENT |
258 | 1 | Dmitry Chernyak | |
259 | 1 | Dmitry Chernyak | Edit @/etc/sci/sci.conf@ |
260 | 1 | Dmitry Chernyak | |
261 | 8 | Владимир Ипатов | Most of values rely of your network setup. In section network setup it was described for most cases. |
262 | 8 | Владимир Ипатов | |
263 | 8 | Владимир Ипатов | Here is additional notes about sci.conf configuring: |
264 | 8 | Владимир Ипатов | |
265 | 1 | Dmitry Chernyak | * You should specify node1 and node2 data as you have installed them. |
266 | 1 | Dmitry Chernyak | *NOTE*: You can setup the cluster even with one node. In this case just leave NODE2_ |
267 | 1 | Dmitry Chernyak | lines as is. In fact this is a dangerous setup, so you will be warned about this duging |
268 | 1 | Dmitry Chernyak | the procedures. |
269 | 1 | Dmitry Chernyak | |
270 | 1 | Dmitry Chernyak | * You should specify the cluster's name and IP. |
271 | 1 | Dmitry Chernyak | |
272 | 1 | Dmitry Chernyak | * NODE#_SAN_IP should be specified on both nodes or none. |
273 | 1 | Dmitry Chernyak | |
274 | 8 | Владимир Ипатов | * NODE#_LAN_IP should be specified on both nodes or none. |
275 | 8 | Владимир Ипатов | |
276 | 1 | Dmitry Chernyak | * If you haven't Internet uplink or have a local package mirrors, you should correct |
277 | 1 | Dmitry Chernyak | APT_ - settings. |
278 | 1 | Dmitry Chernyak | |
279 | 6 | Владимир Ипатов | * If you need to uplink to the DNS hierarchy other than root hint zones, specify DNS_FORWARDERS |
280 | 6 | Владимир Ипатов | (note trailing ';'). |
281 | 1 | Dmitry Chernyak | |
282 | 8 | Владимир Ипатов | * MASTER_NETDEV - master interface name for cluster address. Auto-detected by default. |
283 | 6 | Владимир Ипатов | |
284 | 8 | Владимир Ипатов | * LAN_NETDEV - Network interface to bind to virtual machies by default. Auto-detected by default. |
285 | 6 | Владимир Ипатов | |
286 | 6 | Владимир Ипатов | * RESERVED_VOLS - list of volumes ignored by ganeti. Comma separated. You must specify vg for all volumes in this list. |
287 | 6 | Владимир Ипатов | |
288 | 1 | Dmitry Chernyak | |
289 | 1 | Dmitry Chernyak | h2. SETUP CLUSTER |
290 | 1 | Dmitry Chernyak | |
291 | 1 | Dmitry Chernyak | Issue: |
292 | 1 | Dmitry Chernyak | |
293 | 1 | Dmitry Chernyak | <pre> |
294 | 1 | Dmitry Chernyak | # sci-setup cluster |
295 | 1 | Dmitry Chernyak | </pre> |
296 | 1 | Dmitry Chernyak | |
297 | 1 | Dmitry Chernyak | Check and confirm settings printed. |
298 | 1 | Dmitry Chernyak | |
299 | 1 | Dmitry Chernyak | The process will go on. |
300 | 1 | Dmitry Chernyak | |
301 | 1 | Dmitry Chernyak | Next you will be prompted to accept ssh key from node2 and for the root's password to node2. |
302 | 1 | Dmitry Chernyak | |
303 | 1 | Dmitry Chernyak | On finish you will look something like this: |
304 | 1 | Dmitry Chernyak | |
305 | 1 | Dmitry Chernyak | <pre> |
306 | 1 | Dmitry Chernyak | Verify |
307 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:10 2011 * Verifying global settings |
308 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:10 2011 * Gathering data (1 nodes) |
309 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Verifying node status |
310 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Verifying instance status |
311 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Verifying orphan volumes |
312 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Verifying orphan instances |
313 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Verifying N+1 Memory redundancy |
314 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Other Notes |
315 | 1 | Dmitry Chernyak | Wed Jan 12 15:36:11 2011 * Hooks Results |
316 | 1 | Dmitry Chernyak | Node DTotal DFree MTotal MNode MFree Pinst Sinst |
317 | 1 | Dmitry Chernyak | gnt1.ganeti.example.org 100.0G 100.0G 1020M 379M 625M 0 0 |
318 | 1 | Dmitry Chernyak | gnt2.ganeti.example.org 100.0G 100.0G 1020M 379M 625M 0 0 |
319 | 1 | Dmitry Chernyak | If all is ok, proceed with /usr/local/sbin/sci-setup service |
320 | 1 | Dmitry Chernyak | </pre> |
321 | 1 | Dmitry Chernyak | |
322 | 1 | Dmitry Chernyak | h2. SETUP SERVICE INSTANCE |
323 | 1 | Dmitry Chernyak | |
324 | 1 | Dmitry Chernyak | The service instance is named 'sci' and have a few aliases. |
325 | 1 | Dmitry Chernyak | On setup, it's IP address is determined from @/etc/resolv.conf@ of your first node. |
326 | 1 | Dmitry Chernyak | This instance will be hardcoded in @/etc/hosts@ file of all cluster nodes and instances. |
327 | 1 | Dmitry Chernyak | |
328 | 1 | Dmitry Chernyak | Issue: |
329 | 1 | Dmitry Chernyak | |
330 | 1 | Dmitry Chernyak | <pre> |
331 | 1 | Dmitry Chernyak | # sci-setup service |
332 | 1 | Dmitry Chernyak | </pre> |
333 | 1 | Dmitry Chernyak | |
334 | 1 | Dmitry Chernyak | You'll see the progress of DRBD syncing disks, then the message |
335 | 1 | Dmitry Chernyak | <pre> |
336 | 1 | Dmitry Chernyak | * running the instance OS create scripts... |
337 | 1 | Dmitry Chernyak | </pre> |
338 | 1 | Dmitry Chernyak | appears. The further may take a while. The process finishes with |
339 | 1 | Dmitry Chernyak | <pre> |
340 | 1 | Dmitry Chernyak | * starting instance... |
341 | 1 | Dmitry Chernyak | </pre> |
342 | 1 | Dmitry Chernyak | message. |
343 | 1 | Dmitry Chernyak | |
344 | 1 | Dmitry Chernyak | Now you can log on to the sci instance using: |
345 | 1 | Dmitry Chernyak | |
346 | 1 | Dmitry Chernyak | <pre> |
347 | 1 | Dmitry Chernyak | # gnt-instance console sci |
348 | 1 | Dmitry Chernyak | </pre> |
349 | 1 | Dmitry Chernyak | |
350 | 1 | Dmitry Chernyak | Log in as root, the password is empty. |
351 | 1 | Dmitry Chernyak | *NOTE*: Due to empty password all remote connections to new instance is prohibited. |
352 | 1 | Dmitry Chernyak | You should change password and install @openssh-server@ package manually after |
353 | 1 | Dmitry Chernyak | successful bootstrap procedure. |
354 | 1 | Dmitry Chernyak | |
355 | 1 | Dmitry Chernyak | h2. SERVICE INSTANCE BOOTSTRAP |
356 | 1 | Dmitry Chernyak | |
357 | 1 | Dmitry Chernyak | The system will setup itself via puppet. This is the iterative process. You can monitor |
358 | 1 | Dmitry Chernyak | it by looking into @/var/log/daemon.log@. At start there is no @less@ command yet, so |
359 | 1 | Dmitry Chernyak | you can use @more@, @cat@, @tail@ or @tail -f@ until @less@ will be auto-installed. |
360 | 1 | Dmitry Chernyak | |
361 | 1 | Dmitry Chernyak | By default the iterations are repeated in 20 minutes. To shorten the wait time you can |
362 | 1 | Dmitry Chernyak | issue |
363 | 1 | Dmitry Chernyak | |
364 | 1 | Dmitry Chernyak | <pre> |
365 | 1 | Dmitry Chernyak | # /etc/init.d/puppet restart |
366 | 1 | Dmitry Chernyak | </pre> |
367 | 1 | Dmitry Chernyak | |
368 | 1 | Dmitry Chernyak | and then look into @daemon.log@ how it finishes. |
369 | 1 | Dmitry Chernyak | |
370 | 1 | Dmitry Chernyak | Repeat this a few times until puppet will do nothing in turn. |
371 | 1 | Dmitry Chernyak | |
372 | 1 | Dmitry Chernyak | h2. PREPARING FOR NEW INSTANCES |
373 | 1 | Dmitry Chernyak | |
374 | 1 | Dmitry Chernyak | New instances are created just by regular Ganeti commands such as: |
375 | 1 | Dmitry Chernyak | |
376 | 1 | Dmitry Chernyak | <pre> |
377 | 1 | Dmitry Chernyak | gnt-instance add -t drbd -o debootstrap+default -s 10g -B memory=256m -n NODE1_NAME:NODE2_NAME INSTANCE_NAME |
378 | 1 | Dmitry Chernyak | </pre> |
379 | 1 | Dmitry Chernyak | |
380 | 1 | Dmitry Chernyak | Altought, some tuning hooks are provided by SCI-CD project: |
381 | 1 | Dmitry Chernyak | # Each instance has installed @puppet@ for autoconfiguration and @openssh-client@ for file transfers etc. |
382 | 1 | Dmitry Chernyak | # The instance uses pygrub to boot kernel from /vmlinuz & Co on the innstance's own disk. |
383 | 1 | Dmitry Chernyak | # The instance's network interfaces may be set up automatically as described below. |
384 | 1 | Dmitry Chernyak | |
385 | 1 | Dmitry Chernyak | h3. INSTANCE INTERFACE AUTOCONFIGURATION |
386 | 1 | Dmitry Chernyak | |
387 | 1 | Dmitry Chernyak | If your instances may sit on several networks and you need static addressing in them, you should fulfill |
388 | 1 | Dmitry Chernyak | the file @/etc/ganeti/networks@ with all known networks you want to attach your instances. |
389 | 1 | Dmitry Chernyak | Each line in the file has format |
390 | 1 | Dmitry Chernyak | |
391 | 1 | Dmitry Chernyak | |NETWORK|NETMASK|BROADCAST|GATEWAY| |
392 | 1 | Dmitry Chernyak | |
393 | 1 | Dmitry Chernyak | Ganeti instance debootstrap hook looks in this file for the network, mathing the address of bootstraped |
394 | 1 | Dmitry Chernyak | instance and fulfill it's @/etc/network/interfaces@ accordingly. |
395 | 1 | Dmitry Chernyak | |
396 | 1 | Dmitry Chernyak | *NOTE*: If you have only one default network, you shouldn't care because it's data are preinstalled. |
397 | 1 | Dmitry Chernyak | *NOTE*: networks file must be copied to all cluster nodes (not automated yet). |
398 | 1 | Dmitry Chernyak | |
399 | 1 | Dmitry Chernyak | h2. SCI OPERATIONS |
400 | 1 | Dmitry Chernyak | |
401 | 1 | Dmitry Chernyak | Read [[OPERATIONS]] next. |