My realtime RTT is usually around 150-160ms. However, current realtime RTT (260ms) is higher that than the calculated RTT (157.1). I have tested using basic, advanced and multi path modes but the result is the same. Below are the screenshots for your reference.
I’m using Mudfish on Linux if it helps. Full VPN mode doesn’t seem to work and I lose internet access. I’ve tested running traceroute while basic, advanced or multi path modes were enabled and it seems like the routes taken are very similar to the routes when Mudfish is turned off. My guess is the traceroute is going through my regular interface instead of the interface Mudfish uses. Forcing traceroute to use the tap0 interface returns a timeout for all hops. Interestingly, everything works normally if I use Mudfish 5.4.7 instead of 5.4.8.
Are you meaning v5.4.6? At this moment v5.4.7 is latest one. And what linux distribution do you use?
Sorry, yes I meant Mudfish works normally on 5.4.6 and not on 5.4.7. I’m running Debian Testing on my machine.
Not easy to understand why. Could you please show me the result of netstat -nr when you enabled Full VPN mode?
I think it’s a routing table issue so I need to check it first.
I’m having the same issues since I updated mine.
@seeaye Thank you for this result. When I checked it, it seems the routing table looks good. If I checked it correctly, it seems you tested Full VPN mode with “FR Europe (Gravelines - OVH)” mudfish node, right?
Then did you ping to 5.196.224.179 IP (the public IP of FR Europe (Gravelines - OVH)) and works fine?
I’ve tested full VPN while connected to FR Europe (Gravelines - OVH) and pinging the IP. Below are the results. I’ve also tried connecting to another VPN node and experienced similar results as the ones below to rule out issues with the particular node.
Mudfish 5.4.8 (No internet access and pinging 5.196.224.179 does not work)
Mudfish 5.4.7 (No internet access and pinging 5.196.224.179 does not work)
Mudfish 5.4.6 (Able to access the internet and pinging 5.196.224.179 works)
Definitely the routing issue. It seems the routing table isn’t set properly if Full VPN mode is ON.
If I tried to reproduce this issue on Ubuntu 20.04 LTS, it looks it’s fine with it. However the problem on your box is that running /usr/sbin/route add -net <IP> netmask <netmask> gw <gateway> is failing due to unknown reason… So could you please show me the outputs of the following command while Full VPN is enabled and no internet issue is happening?
cat /opt/mudfish/5.4.8/var/mudfish_stdout.txtps auxww |grep mudfis
It seems the command execution process of mudfish core is abnormally crashed or doesn’t work.
cat /opt/mudfish/5.4.8/var/mudfish_stdout.txt
ps auxww |grep mudfis
root 54574 0.0 0.0 10556 4848 pts/1 S+ 21:16 0:00 sudo /opt/mudfish/5.4.8/bin/mudrun
root 54575 0.1 0.0 12244 5356 pts/1 Sl+ 21:16 0:00 /opt/mudfish/5.4.8/bin/mudrun
root 54576 0.0 0.0 3780 560 pts/1 Sl+ 21:16 0:00 /opt/mudfish/5.4.8/bin/mudrun
root 55648 0.0 0.0 2420 584 pts/1 S+ 21:17 0:00 /bin/sh -c /opt/mudfish/5.4.8/bin/mudflow -r -p rttmethod=UDP -n -s 451 -c -v -v -v -v
root 55650 0.0 0.0 2420 524 pts/1 S+ 21:17 0:00 /bin/sh -c /opt/mudfish/5.4.8/bin/mudfish -R -u seeaye -p '<REDACTED>' -F -l -T auto -f -s 451 -M 1400
root 55652 0.1 0.0 7580 2816 pts/1 Sl+ 21:17 0:00 /opt/mudfish/5.4.8/bin/mudflow -r -p rttmethod UDP -n -s 451 -c -v -v -v -v
root 55653 9.5 0.0 28492 10884 pts/1 Sl+ 21:17 0:25 /opt/mudfish/5.4.8/bin/mudfish -R -u seeaye -p <REDACTED> -F -l -T auto -f -s 451 -M 1400
root 55654 0.0 0.0 1804 488 pts/1 Sl+ 21:17 0:00 /opt/mudfish/5.4.8/bin/mudfish -R -u seeaye -p <REDACTED> -F -l -T auto -f -s 451 -M 1400
root 56906 0.0 0.0 2420 580 pts/1 S+ 21:17 0:00 /bin/sh -c dhclient -d -pf /var/run/dhclient-mudfish.pid tap0
root 56908 0.0 0.0 99824 6432 pts/1 Sl+ 21:17 0:00 dhclient -d -pf /var/run/dhclient-mudfish.pid tap0
aizwrai+ 58547 0.0 0.0 6184 664 pts/3 S+ 21:22 0:00 grep mudfisFrom the logs it looks like Mudfish 5.4.8 is unable to find the route binary in /usr/sbin/ which might be causing the routing issues. I ran which route on my machine and route is in /sbin/. The logs for 5.4.6 seem to show that it looks for route in /sbin which is why Mudfish in that version appears to work for me.
cat /opt/mudfish/5.4.6/var/mudfish_stdout.txt
Mon, 11 Oct 2021 13:34:19 GMT [0.076747] [INFO] mudfish v5.4.6 (for amd64)
Mon, 11 Oct 2021 13:34:19 GMT [0.076792] [INFO] Timestamp: Mon Oct 11 21:34:19 2021
Mon, 11 Oct 2021 13:34:19 GMT [0.076799] [INFO] Full VPN mode is enabled
Mon, 11 Oct 2021 13:34:19 GMT [0.076808] [INFO] Connection protocol: auto
Mon, 11 Oct 2021 13:34:19 GMT [0.076814] [INFO] Authenticating...
Mon, 11 Oct 2021 13:34:20 GMT [0.255438] [INFO] Fetching the static nodes...
Mon, 11 Oct 2021 13:34:21 GMT [1.335621] [INFO] Fetched 557 static nodes from the master.
Mon, 11 Oct 2021 13:34:21 GMT [1.335927] [INFO] New tap(4) interface is created: tap0
Mon, 11 Oct 2021 13:34:21 GMT [1.354103] [INFO] No load calculation for clock.
Mon, 11 Oct 2021 13:34:21 GMT [1.404658] [INFO] Uses the macaddr for msk(4) (cached).
Mon, 11 Oct 2021 13:34:21 GMT [1.404742] [INFO] msk0: Ethernet address: 16:ef:3f:f1:3e:af
Mon, 11 Oct 2021 13:34:21 GMT [1.406951] [INFO] Fetching new static node SID 451...
Mon, 11 Oct 2021 13:34:21 GMT [1.694715] [INFO] re367: Ethernet address: 6a:46:3e:56:a7:44
Mon, 11 Oct 2021 13:34:21 GMT [1.862180] [INFO] re367: Testing to ip 5.196.224.179 port 10006 using UDP (good)
Mon, 11 Oct 2021 13:34:21 GMT [1.862222] [INFO] re367: Broker feature is enabled.
Mon, 11 Oct 2021 13:34:21 GMT [1.863346] [INFO] Setting the default gateway as 10.230.96.1 (451 (null))
Mon, 11 Oct 2021 13:34:21 GMT [1.863557] [INFO] Ready
Mon, 11 Oct 2021 13:34:21 GMT [1.863631] [INFO] udp_relay: listening on 0.0.0.0:1 (fd 0)
Mon, 11 Oct 2021 13:34:21 GMT [1.863689] [INFO] tcp_relay: listening on 0.0.0.0:1 (fd 0)
Mon, 11 Oct 2021 13:34:21 GMT [1.863826] [INFO] Waiting until the client's ready to handle the routing path list.
Mon, 11 Oct 2021 13:34:21 GMT [1.864833] [INFO] re367: Connected to 5.196.224.179:10006 (UDP)
Mon, 11 Oct 2021 13:34:21 GMT [1.905207] [INFO] Listening the management console at 127.0.0.1:36011
Mon, 11 Oct 2021 13:34:24 GMT [4.864159] [INFO] Waiting until the client's ready to handle the routing path list.
Mon, 11 Oct 2021 13:34:25 GMT [5.405654] [INFO] Commander said: SIOCADDRT: File exists
Mon, 11 Oct 2021 13:34:25 GMT [5.405807] [INFO] Commander said: CMD !/sbin/route add -net 5.196.224.179 netmask 255.255.255.255 gw 192.168.1.254 R 61060 Status: 0700 Bad exit code: 0700 sig 0 exit 7
Mon, 11 Oct 2021 13:34:25 GMT [5.451674] [INFO] Commander said: Internet Systems Consortium DHCP Client 4.4.1
Copyright 2004-2018 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Mon, 11 Oct 2021 13:34:25 GMT [5.489347] [INFO] Commander said: Listening on LPF/tap0/de:b1:e3:18:4a:eb
Mon, 11 Oct 2021 13:34:25 GMT [5.489489] [INFO] Commander said: Sending on LPF/tap0/de:b1:e3:18:4a:eb
Mon, 11 Oct 2021 13:34:25 GMT [5.489625] [INFO] Commander said: Sending on Socket/fallback
Mon, 11 Oct 2021 13:34:25 GMT [5.489718] [INFO] Commander said: DHCPREQUEST for 10.254.161.175 on tap0 to 255.255.255.255 port 67
Mon, 11 Oct 2021 13:34:25 GMT [5.489734] [INFO] DHCP request for de:b1:e3:18:4a:eb (nortrom) from interface
Mon, 11 Oct 2021 13:34:25 GMT [5.489917] [INFO] Commander said: DHCPNAK from 10.254.165.1
Mon, 11 Oct 2021 13:34:25 GMT [5.524880] [INFO] DHCP discover for de:b1:e3:18:4a:eb (nortrom) from interface
Mon, 11 Oct 2021 13:34:25 GMT [5.524930] [INFO] DNS address: 10.230.96.1
Mon, 11 Oct 2021 13:34:25 GMT [5.524888] [INFO] Commander said: DHCPDISCOVER on tap0 to 255.255.255.255 port 67 interval 3
Mon, 11 Oct 2021 13:34:25 GMT [5.525222] [INFO] Commander said: DHCPOFFER of 10.254.165.89 from 10.254.165.1
Mon, 11 Oct 2021 13:34:25 GMT [5.525252] [INFO] DHCP request for de:b1:e3:18:4a:eb (nortrom) from interface
Mon, 11 Oct 2021 13:34:25 GMT [5.525264] [INFO] DNS address: 10.230.96.1
Mon, 11 Oct 2021 13:34:25 GMT [5.525285] [INFO] DHCP Client got IPv4 address now
Mon, 11 Oct 2021 13:34:25 GMT [5.525295] [INFO] Commander said: DHCPREQUEST for 10.254.165.89 on tap0 to 255.255.255.255 port 67
Mon, 11 Oct 2021 13:34:25 GMT [5.525519] [INFO] Commander said: DHCPACK of 10.254.165.89 from 10.254.165.1
Mon, 11 Oct 2021 13:34:25 GMT [5.766955] [INFO] Commander said: bound to 10.254.165.89 -- renewal in 2158003 seconds.ps auxww |grep mudfis
root 59508 0.0 0.0 10556 4984 pts/1 S+ 21:34 0:00 sudo /opt/mudfish/5.4.6/bin/mudrun
root 59509 0.0 0.0 11400 4464 pts/1 Sl+ 21:34 0:00 /opt/mudfish/5.4.6/bin/mudrun
root 59510 0.0 0.0 3260 488 pts/1 Sl+ 21:34 0:00 /opt/mudfish/5.4.6/bin/mudrun
root 59758 0.0 0.0 2420 588 pts/1 S+ 21:34 0:00 /bin/sh -c /opt/mudfish/5.4.6/bin/mudflow -r -p rttmethod=UDP -n -s 451 -c -v -v -v -v
root 59760 0.0 0.0 2420 588 pts/1 S+ 21:34 0:00 /bin/sh -c /opt/mudfish/5.4.6/bin/mudfish -R -u seeaye -p '<REDACTED>' -F -l -T auto -f -s 451 -M 1400
root 59762 0.5 0.0 22772 13028 pts/1 Sl+ 21:34 0:03 /opt/mudfish/5.4.6/bin/mudfish -R -u seeaye -p <REDACTED> -F -l -T auto -f -s 451 -M 1400
root 59763 0.1 0.0 7484 3016 pts/1 Sl+ 21:34 0:01 /opt/mudfish/5.4.6/bin/mudflow -r -p rttmethod UDP -n -s 451 -c -v -v -v -v
root 59765 0.0 0.0 1780 440 pts/1 Sl+ 21:34 0:00 /opt/mudfish/5.4.6/bin/mudfish -R -u seeaye -p <REDACTED> -F -l -T auto -f -s 451 -M 1400
root 61076 0.0 0.0 2420 524 pts/1 S+ 21:34 0:00 /bin/sh -c dhclient -d -pf /var/run/dhclient-mudfish.pid tap0
root 61078 0.0 0.0 99824 6448 pts/1 Sl+ 21:34 0:00 dhclient -d -pf /var/run/dhclient-mudfish.pid tap0
aizwrai+ 67472 0.0 0.0 6184 664 pts/4 S+ 21:45 0:00 grep mudfis Ooops… I found a reason… The reason is a path of route command. In the previous it was /sbin/route. and nowadays it’s changed to /usr/sbin/route.
Please run the following command and test again:
sudo ln -s /sbin/route /usr/sbin/routeI ran sudo ln -s /sbin/route /usr/sbin/route and mudfish still had the same issue. I checked the logs in /opt/mudfish/5.4.8/var/mudfish_stdout.txt and seems looks like mudfish couldn’t find the ip executable in /usr/sbin/.
I created a symlink for ip as well and all seems to be good. So to summarize I created the following symlinks:
sudo ln -s /sbin/route /usr/sbin/route
sudo ln -s /sbin/ip /usr/sbin/ip
Maybe for the next mudfish release it would be good if mudfish checks the locations of executables it needs on the user’s system. Assuming paths might result in edge cases like I faced.
1 Like
Cool! Thank you for feedback. I’d patched your problem in the code and will be available at next release. At that moment I can remove unnecessary symbolic links. 
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.