The UltraESB has built-in capability of communicating with FTP, FTP/S, SFTP and local filesystem (file://) endpoints. While setting up the other file transports might be easy, FTP/S can be problematic, especially if you start from a fresh FTP/S installation.
Assuming a local FTP/S server setup (e.g. vsftpd) with a user ftp having password ftp and home directory /srv/ftp (quoting vsftpd default settings), you might encounter a few problems when trying to submit files to an endpoint URL such as ftps://ftp:ftp@localhost/srv/ftp/, on the UESB side.
These usually occur at FTP level, so in order to find the actual causes you will have to enable TRACE logging on the org.adroitlogic.ultraesb.transport.file category in $ULTRA_HOME/conf/log4j.properties:
log4j.category.org.adroitlogic.ultraesb.transport.file=TRACEWith TRACE enabled as above, the UESB log will contain a trace of all FTP commands being executed during the transfer attempt, as:
2016-05-04 22:57:55,043 [-] [primary-1] [system] [000000I] DEBUG FTPSConnector Connecting over FTPS to URL : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no 220 (vsFTPd 3.0.3) AUTH TLS 234 Proceed with negotiation. PROT P 200 PROT now Private. USER ftp ...
If you see 530 Anonymous sessions may not use encryption. (usually followed by a stacktrace on the UESB):
... 234 Proceed with negotiation. PROT P 200 PROT now Private. USER ftp 530 Anonymous sessions may not use encryption. QUIT 221 Goodbye. 2016-05-04 22:57:55,381 [-] [primary-1] [system] [000000W] WARN FTPSConnector Login failed for FTPS server : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no PORT 127,0,0,1,179,254 2016-05-04 22:57:55,385 [-] [primary-1] [system] [000000W] WARN FTPConnector Error copying file to URL : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no org.apache.commons.net.ftp.FTPConnectionClosedException: Connection closed without indication. at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:313) at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:290) at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479) at org.apache.commons.net.ftp.FTPSClient.sendCommand(FTPSClient.java:535) ...
you would want to allow anonymous SSL sessions on your vsftpd.conf:
allow_anon_ssl=YES(Note: Make sure you restart vsftpd after every such change!
service vsftpd restart # as root)
If the issue is 522 SSL connection failed: session reuse required:
... 234 Proceed with negotiation. PROT P 200 PROT now Private. USER ftp 331 Please specify the password. PASS ftp 230 Login successful. TYPE I 200 Switching to Binary mode. PASV 227 Entering Passive Mode (127,0,0,1,161,222). STOR /srv/ftp/ftps/send.msg 150 Ok to send data. 522 SSL connection failed: session reuse required 2016-05-04 22:59:07,418 [-] [primary-1] [system] [000000I] DEBUG FTPConnector FTP copy failed to : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no 2016-05-04 22:59:07,429 [-] [primary-1] [system] [000000W] WARN FileTransportSender Error sending message via file transport sender to : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no
adding/updating require_ssl_reuse=NO in vsftpd.conf would resolve it.
On the other hand, if authentication fails mid-way in the SSL handshake process:
220 (vsFTPd 3.0.3) AUTH TLS 234 Proceed with negotiation. 2016-05-04 23:05:26,907 [-] [primary-2] [system] [000000W] WARN FTPSConnector Error connecting over FTPS to URL : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.Alerts.getSSLException(Alerts.java:154) ... at org.adroitlogic.ultraesb.transport.file.FTPSConnector.(FTPSConnector.java:62) at org.adroitlogic.ultraesb.transport.file.FileSystemConnectorFactory.createConnector(FileSystemConnectorFactory.java:54) ... at org.adroitlogic.ultraesb.core.work.SimpleQueueWorkManager$1.run(SimpleQueueWorkManager.java:306) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2016-05-04 23:05:26,909 [-] [primary-2] [system] [000000W] WARN FTPConnector Error copying file to URL : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no java.net.SocketException: Socket closed at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116) at java.net.SocketOutputStream.write(SocketOutputStream.java:153) ... at org.apache.commons.net.ftp.FTP.__send(FTP.java:501) at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:475) ... at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1795) at org.adroitlogic.ultraesb.transport.file.FTPConnector.put(FTPConnector.java:169) at org.adroitlogic.ultraesb.transport.file.FileTransportSender.internalSend(FileTransportSender.java:80) ... at org.adroitlogic.ultraesb.core.work.SimpleQueueWorkManager$1.run(SimpleQueueWorkManager.java:306) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2016-05-04 23:05:26,919 [-] [primary-2] [system] [000000W] WARN FileTransportSender Error sending message via file transport sender to : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no
you might require a tcpdump (against the loopback interface (lo) in this case) for finding the cause, as neither the ESB logs (even with SSL debug logs enabled: -Djavax.net.debug=all) nor the vsftpd log would show the underlying cause properly:
# tcpdump -A -vvv -i lo
...
06:41:37.566325 IP (tos 0x0, ttl 64, id 14724, offset 0, flags [DF], proto TCP (6), length 83)
localhost.ftp > localhost.60532: Flags [P.], cksum 0xfe47 (incorrect -> 0xc3fd), seq 21:52, ack 11, win 342, options [nop,nop,TS val 37556708 ecr 37556708], length 31: FTP, length: 31
234 Proceed with negotiation.
E..S9.@.@..............t..(u..,....V.G.....
.=...=..234 Proceed with negotiation.
06:41:37.576134 IP (tos 0x0, ttl 64, id 15677, offset 0, flags [DF], proto TCP (6), length 294)
localhost.60532 > localhost.ftp: Flags [P.], cksum 0xff1a (incorrect -> 0x9887), seq 11:253, ack 52, win 342, options [nop,nop,TS val 37556711 ecr 37556708], length 242: FTP, length: 242
E..&==@.@............t....,...(....V.......
.=...=.............W*...l....I...TG R.1_........F>...d.$.(.=.&.*.k.j.
...5.....9.8.#.'.<.%.).g.@. .../.....3.2.,.+.0.....2...../...-.1.........
.............\.
.4.2............... .
....................................................................
06:41:37.576284 IP (tos 0x0, ttl 64, id 14725, offset 0, flags [DF], proto TCP (6), length 59)
localhost.ftp > localhost.60532: Flags [P.], cksum 0xfe2f (incorrect -> 0xd702), seq 52:59, ack 253, win 350, options [nop,nop,TS val 37556711 ecr 37556711], length 7: FTP, length: 7
E..;9.@.@..6...........t..(...-....^./.....
.=...=........(
06:41:37.577983 IP (tos 0x0, ttl 64, id 14726, offset 0, flags [DF], proto TCP (6), length 62)
localhost.ftp > localhost.60532: Flags [P.], cksum 0xfe32 (incorrect -> 0xd9ea), seq 59:69, ack 253, win 350, options [nop,nop,TS val 37556711 ecr 37556711], length 10: FTP, length: 10
500 OOPS: [!ftp]
E..>9.@.@..2...........t..(...-....^.2.....
.=...=..500 OOPS:
06:41:37.577995 IP (tos 0x0, ttl 64, id 14727, offset 0, flags [DF], proto TCP (6), length 118)
localhost.ftp > localhost.60532: Flags [P.], cksum 0xfe6a (incorrect -> 0xe009), seq 69:135, ack 253, win 350, options [nop,nop,TS val 37556711 ecr 37556711], length 66: FTP, length: 66
error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher[!ftp]
E..v9.@.@..............t..(...-....^.j.....
.=...=..error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher
06:41:37.578000 IP (tos 0x0, ttl 64, id 14728, offset 0, flags [DF], proto TCP (6), length 54)
localhost.ftp > localhost.60532: Flags [P.], cksum 0xfe2a (incorrect -> 0x0bb0), seq 135:137, ack 253, win 350, options [nop,nop,TS val 37556711 ecr 37556711], length 2: FTP, length: 2
E..69.@.@..8...........t..(...-....^.*.....
.=...=..
06:41:37.578230 IP (tos 0x0, ttl 64, id 14729, offset 0, flags [DF], proto TCP (6), length 62)
localhost.ftp > localhost.60532: Flags [P.], cksum 0xfe32 (incorrect -> 0xd99c), seq 137:147, ack 253, win 350, options [nop,nop,TS val 37556711 ecr 37556711], length 10: FTP, length: 10
500 OOPS: [!ftp]
E..>9.@.@../...........t..(...-....^.2.....
.=...=..500 OOPS:
06:41:37.578323 IP (tos 0x0, ttl 64, id 15678, offset 0, flags [DF], proto TCP (6), length 52)
localhost.60532 > localhost.ftp: Flags [R.], cksum 0xfe28 (incorrect -> 0x18bc), seq 253, ack 147, win 342, options [nop,nop,TS val 37556711 ecr 37556711], length 0
E..4=>@.@............t....-...(....V.(.....
.=...=..
Now we have a hint on what might be going wrong:
error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher[!ftp]This usually occurs if the cipher suite offered by the FTP/S server is not sufficiently strong. Adding/updating
ssl_ciphers=HIGHin vsftpd.conf would get you through this issue.
Once you have everything in place, the UESB trace would indicate a successful transfer:
2016-05-04 23:02:08,930 [-] [primary-1] [system] [000000I] DEBUG FTPSConnector Connecting over FTPS to URL : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no 220 (vsFTPd 3.0.3) AUTH TLS 234 Proceed with negotiation. PROT P 200 PROT now Private. USER ftp 331 Please specify the password. PASS ftp 230 Login successful. TYPE I 200 Switching to Binary mode. PASV 227 Entering Passive Mode (127,0,0,1,104,213). STOR /srv/ftp/ftps/send.msg 150 Ok to send data. 226 Transfer complete. 2016-05-04 23:02:09,360 [-] [primary-1] [system] [000000I] DEBUG FTPConnector FTP copy completed successfully to : ftps://ftp:********@localhost/srv/ftp/ftps/send.msg?StrictHostKeyChecking=no
