12.4. FTP In an LVS Cluster
File Transport Protocol (FTP) is an old and complex multi-port protocol that presents a distinct set of challenges to a clustered environment. To understand the nature of these challenges, you must first understand some key things about how FTP works.
12.4.1. How FTP Works
With most other server client relationships, the client machine opens up a connection to the server on a particular port and the server then responds to the client on that port. When an FTP client connects to an FTP server it opens a connection to the FTP control port 21. Then the client tells the FTP server whether to establish an active or passive connection. The type of connection chosen by the client determines how the server responds and on what ports transactions will occur.
The two types of data connections are:
- Active Connections
When an active connection is established, the server opens a data connection to the client from port 20 to a high range port on the client machine. All data from the server is then passed over this connection.
- Passive Connections
When a passive connection is established, the client asks the FTP server to establish a passive connection port, which can be on any port higher than 10,000. The server then binds to this high-numbered port for this particular session and relays that port number back to the client. The client then opens the newly bound port for the data connection. Each data request the client makes results in a separate data connection. Most modern FTP clients attempt to establish a passive connection when requesting data from servers.
The two important things to note about all of this in regards to clustering is:
The client determines the type of connection, not the server. This means, to effectively cluster FTP, you must configure the LVS routers to handle both active and passive connections.
The FTP client/server relationship can potentially open a large number of ports that the Piranha Configuration Tool and IPVS do not know about.
12.4.2. How This Affects LVS Routing
IPVS packet forwarding only allows connections in and out of the cluster based on it recognizing its port number or its firewall mark. If a client from outside the cluster attempts to open a port IPVS is not configured to handle, it drops the connection. Similarly, if the real server attempts to open a connection back out to the Internet on a port IPVS does not know about, it drops the connection. This means all connections from FTP clients on the Internet must have the same firewall mark assigned to them and all connections from the FTP server must be properly forwarded to the Internet using network packet filtering rules.
12.4.3. Creating Network Packet Filter Rules
Before assigning any iptables rules for FTP service, review the information in Section 12.3.1 Assigning Firewall Marks concerning multi-port services and techniques for checking the existing network packet filtering rules.
Below are rules which assign the same firewall mark, 21, to FTP traffic. For these rules to work properly, you must also use the VIRTUAL SERVER subsection of Piranha Configuration Tool to configure a virtual server for port 21 with a value of 21 in the Firewall Mark field. See Section 13.6.1 The VIRTUAL SERVER Subsection for details.
12.4.3.1. Rules for Active Connections
The rules for active connections tell the kernel to accept and forward connections coming to the internal floating IP address on port 20 — the FTP data port.
- iptables
/sbin/iptables -t nat -A POSTROUTING -p tcp \ -s n.n.n.0/24 --sport 20 -j MASQUERADE
In the above iptables commands, n.n.n should be replaced with the first three values for the floating IP for the NAT interface's internal network interface defined in the GLOBAL SETTINGS panel of Piranha Configuration Tool. The command allows the LVS router to accept outgoing connections from the real servers that IPVS does not know about.
12.4.3.2. Rules for Passive Connections
The rules for passive connections assign the appropriate firewall mark to connections coming in from the Internet to the floating IP for the service on a wide range of ports — 10,000 to 20,000.
 | Warning | ||
|---|---|---|---|
If you are limiting the port range for passive connections, you must also configure the VSFTP server to use a matching port range. This can be accomplished by adding the following lines to /etc/vsftpd.conf:
You must also control the address that the server displays to the client for passive FTP connections. In a NAT routed LVS system, add the following line to /etc/vsftpd.conf to override the real server IP address to the VIP, which is what the client sees upon connection. For example:
Replace X.X.X.X with the VIP address of the LVS system. For configuration of other FTP servers, consult the respective documentation. |
This range should be a wide enough for most situations; however, you can increase this number to include all available non-secured ports by changing 10000:20000 in the commands below to 1024:65535.
- iptables
/sbin/iptables -t mangle -A PREROUTING -p tcp \ -d n.n.n.n/32 \ --dport 21 -j MARK --set-mark 21 /sbin/iptables -t mangle -A PREROUTING -p tcp \ -d n.n.n.n/32 \ --dport 10000:20000 -j MARK --set-mark 21
In the above iptables commands, n.n.n.n should be replaced with the floating IP for the FTP virtual server defined in the VIRTUAL SERVER subsection of Piranha Configuration Tool. These commands have the net effect of assigning any traffic addressed to the floating IP on the appropriate ports a firewall mark of 21, which is in turn recognized by IPVS and forwarded appropriately.
| Warning |
|---|---|
The commands above take effect immediately, but do not persist through a reboot of the system. To ensure network packet filter settings are restored after a reboot, see Section 12.5 Saving Network Packet Filter Settings |
Finally, you need to be sure that the appropriate service is set to activate on the proper runlevels. For more on this, refer to Section 11.1 Configuring Services on the LVS Routers.