Re: [gentoo-user] Re: replacement for ftp?

[Kent Fredric]

On Tue, 16 May 2017 04:58:44 +0200
Kai Krakow  wrote:

> If this is the underlying (and perfectly legitimate) problem, you need
> to deploy a solution that's most easy for your users and not for you.
> That may involve a custom transfer solution where they simply can drop
> files to. The underlying technology is then up to you: Use what is
> appropriate.

If users have difficulty with keyboard/mouse, but you find a client
that's to their liking, but they're not necessarily capable of
configuring for your service, it might be possible to find a "config
file" for that client which they can fetch over plain ol' http:// with
whatever tools they usually use for downloading http:// things.

If the program they use doesn't have a "launcher config thinger" but
can be imagined to use whatever transfer technology you choose ( ssh ),
and you don't fancy writing a dedicated client, it could be simpler to
write an installer for them, which configures their existing client for
your service.

But I'd imagine lots of confusion is emanating from this list due to
the requirements not being entirely clear/obvious.


pgptIKpyxTiAF.pgp
Description: OpenPGP digital signature

[gentoo-user] Re: replacement for ftp?

[Nikos Chantziaras]

On 04/25/2017 05:29 PM, lee wrote:

since the usage of FTP seems to be declining, what is a replacement
which is at least as good as FTP?

I'm aware that there's webdav, but that's very awkward to use and
missing features.


Is this about security? Then the closest replacement is FTPS (aka SSL 
FTP). It's just plain old FTP at its core, but over SSL. You do need an 
FTP client that supports SSL though. You can configure it to only 
encrypt passwords while the payload data remains non-encrypted.


There's also SFTP, but that's completely different (even though it has 
"FTP" in the name, it's actually SSH, not FTP.)


And there's rsync too, which is for synchronization of files (only 
changes are uploaded, not the whole data) which makes it very fast if 
what you upload is intended to replace old data.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Mon, 15 May 2017 21:31:32 +0100
schrieb lee :

> > I'm sorry, but that's only marginally more believable than claiming
> > keyboards are too complicated for your users.  
> 
> Does it matter what you or I believe?  Some users have difficulties
> using a keyboard and/or a mouse.  I've seen that, so no, what you or I
> believe does not matter.

If this is the underlying (and perfectly legitimate) problem, you need
to deploy a solution that's most easy for your users and not for you.
That may involve a custom transfer solution where they simply can drop
files to. The underlying technology is then up to you: Use what is
appropriate.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Mon, 15 May 2017 21:47:17 +0100
schrieb lee :

> > Depending on what data is transferred, you should also take into
> > account if your solution is certificated to transfer such data. E.g.
> > medical data may only be transferred through properly certificated
> > VPN appliances. Otherwise, you should fall back to sneakernet. I'm
> > not sure how that is any more secure but that's how things are.  
> 
> Interesting, who certifies such appliances?

I really never asked... ;-) Maybe I should...


> What if I, as a patient,
> do not want my data transferred that way,

See your words below: "nobody in Germany actually cares"... So you
won't be asked because it's secure by definition (as in
"certification"). ;-)

The old transport was ISDN. But that is being shut down.

Or did you direct your concern to sneakernet transmission? I doubt that
such data would even be encrypted... Although it clearly should.


> and how do I know if they
> didn't make a mistake when certifying the equipment?

That's German bureaucracy: It has the certificate stamp, so it's okay.
The technical internals do not matter: Nobody asks for that after it's
been certified.


> It's not medical data, and nobody in Germany actually cares about
> protecting peoples data anyway.  The little that is being done towards
> that is nothing but pretense.

We are servicing a medical laboratory: They take this certification
very seriously, so at least they care to fulfill the requirements.
However, we do not control that: After the initial setup they do most
configuration by themselves and we only deliver equipment now. As far
as I know, they cannot even freely choose the provider on their side of
the connection. And they are managing their internal network by
themselves, we wouldn't be easily allowed to do that.

Usually, as a IT service company, you would also sign a non-disclosure
contract when working for a company handling sensitive data. But only
few companies seem to know that...


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Mon, 15 May 2017 22:14:48 +0100
schrieb lee :

> Kai Krakow  writes:
> 
> > Am Sun, 14 May 2017 01:28:55 +0100
> > schrieb lee :
> >  
> >> Kai Krakow  writes:
> >>   
>  [...]  
>  [...]  
> >>  [...]
>  [...]  
> >> 
> >> Wow, you must be living in some sort of paradise.  Here, internet
> >> is more like being cut off from the rest of the world.
> >> 
> >> But then, there's a manufacturer that makes incredibly slow USB
> >> sticks which I won't buy anymore ...  
> >
> > Okay, it really depends. I shouldn't say "most"... ;-)  
> 
> Intenso --- pretty cheap, but awfully slow; however, it does
> work. Better don't buy anything they make unless your time is entirely
> worthless to you.
> 
> > I compared my really crappy (but most reliable yet) old USB stick
> > to my internet connection. My USB stick doesn't do 48 MByte/s, more
> > like 5-10. And don't even ask when writing data.  
> 
> 5--10MB/s?  How do you get that much?

For reading? It can work, tho it will eventually drop to 2 MB/s after a
short time. For writing: It drops well below 1 MB/s after a short burst.

> > Even my rusty hard disk (read: not SSD) has a hard time writing
> > away a big download with constantly high download rate.  
> 
> It must be really old then, about 20 years.

No, it's just that other IO is also ongoing and filesystem internals
have some write overheads and involve head movement which easily limits
the drive from its theoretical ideal rate of 120-150 MB/s. Short
bursts: No problem. Long running writes are more like 50-70 MB/s, which
is pretty near the download rate.

It's also what I see in gigabit networks: Copy speed could be somewhere
between 100 and 120 MB/s, but the local drive seems to easily limit
this to 70-80 MB/s.

My current setup allows constant writing of around 270-280 MB/s
according to:

# dd bs=1M if=/dev/urandom of=test.dat
13128171520 bytes (13 GB, 12 GiB) copied, 48,0887 s, 273 MB/s

So it's not that bad... ;-)

But dd also runs at 100% CPU during that time, so I guess the write
rate could be even higher. I see combined rate of up to 500 MB/s
sometimes tho I'm not sure if this is actual transfer rate or just
queued IO rate. Also, it is pretty near the SATA bus saturation. I'm
not sure if my chipset would deliver this rate per SATA connection or
as a combined rate.


> > But I guess that a good internet connection should be at least 50
> > MBit these days.  
> 
> I'd say 100, but see above.  The advantage is that you have sufficient
> bandwidth to do several things at the same time.  I've never seen fast
> internet.

My provider easily delivers such rates, given the remote side is fast
enough. Most downloads a saturated at around 15-20 MB/s. Only few
servers can deliver more. Probably not only a limit of the servers, but
the peer network connections.


> > And most USB sticks are really crappy at writing. That also counts
> > when you do not transfer the file via network. Of course, most DSL
> > connections have crappy upload speed, too. Only lately, Telekom
> > offers 40 MBit upload connections in Germany.  
> 
> They offer 384kbit/s downstream and deliver 365.  It's almost
> symmetrical, yet almost unusable.

Sounds crappy... No alternative providers there? Problem is almost
always a combination of multiple factors: A long running cable limiting
DSL to a lower physical bandwidth, and usually a too limited traffic
concentrator in that area: You should see very different transfer rates
at different times of the day.


> They also offer 50Mbit and deliver between 2 and 12, and upstream is
> awfully low.  Tell them you could pay for 16 instead of 50 because you
> don't get even that much, and they will tell you that you would get
> even less than you do now.  That is unacceptable.

Yes... They would downgrade you to less performing DSL technology then.
It's all fine for them because you only pay for "up to" that bandwidth.


> And try to get a static IP so you could really use your connection ...

No problems so far, at least for business plans.


> > I'm currently on a 400/25 MBit link and can saturate the link only
> > with proper servers like the Steam network which can deliver 48
> > MByte/s.  
> 
> You must be sitting in a data center and be very lucky to have that.

Cable network... In a smallish city.

Next involvement is announced to be 1 GBit in around 2018-2019...
Something, that's already almost standard in other European countries.
Well... "standard" in terms of availability... Not actual usage. I
think the prices will be pretty high. But that's okay: If you need it,
you should be willing to pay that. It won't help to have such bandwidth
without the provider being able to effort the needed infrastructure.
It's already over-provisioned too much as you already found out.


-- 
Regards,
Kai

Replies to list-only preferred.

Re: [gentoo-user] Re: replacement for ftp?

[Mick]

On Monday 15 May 2017 20:57:50 Kai Krakow wrote:
> > Of course the server will have to be accessible over port 500 for the
> > clients to be able to get to it, but this is a port forwarding/DMZ
> > network configuration exercise at the server end.
> 
> Oh wait... So I need to forward port 500 and 4500 so NAT-T does work
> properly? Even when both sides are NATed? I never got that to work
> reliably for one side NATed, and it never worked for both sides NATed.
> And my research in support forums always said: That does not work...

Well, I haven't presented a complete topology of a proposed network 
architecture because I don't know what the OP's set up is.  I assumed in the 
above statement that the VPN gateway is running on the same (probably NAT'ed) 
server as the ftp service.  Therefore port 500 won't be accessible from the 
Internet unless forwarded.  If the VPN gateway is public facing then no port 
forwarding is necessary.  Site to site IPSec VPN needs only port 500 to set up 
the tunnel.

I have used mobile clients to VPN gateway connections, using IPsec tunnels 
with the client side NAT'ed and the link was very reliable.  Even when the 
mobile clients were connected using unreliable WiFi the tunnel would be re-
established when the WiFi link connectivity was restored.  Key to keeping the 
connection up is to enable Dead-Peer-Detection, or set up some regular ping 
between the peers if either side does not support DPD.  IKEv2 is better than 
IKEv1 and it also allows client roaming (MOBIKE).

Anyway, this is probably getting off topic.  Lee, please start a new thread 
with VPN specific questions if you need help to get it going.  There are quite 
a few examples on the interwebs for configuring OpenVPN and various 
implementations of IKE/IPSec VPNs.  For the latter I recommend StrongSwan 
which has extensive documentation and example configurations.

Saying all this, I would still stick with ftps/filezilla and get the users 
trained.  When things don't work troubleshooting ought to be simpler.  ;-)

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sun, 14 May 2017 01:25:24 +0100
> schrieb lee :
>
>> "Poison BL."  writes:
>> 
>> > On Sat, Apr 29, 2017 at 9:11 PM, lee  wrote:  
>> >>
>> >> "Poison BL."  writes:  
>>  [...]  
>> > trust  
>>  [...]  
>> >>
>> >> Why not?  (12GB are nowhere close to half a petabyte ...)  
>> >
>> > Ah... I completely misread that "or over 50k files in 12GB" as 50k
>> > files *at* 12GB each... which works out to 0.6 PB, incidentally.
>> >  
>> >> The data would come in from suppliers.  There isn't really anything
>> >> going on atm but fetching data once a month which can be like
>> >> 100MB or 12GB or more.  That's because ppl don't use ftp ...  
>> >
>> > Really, if you're pulling it in from third party suppliers, you
>> > tend to be tied to what they offer as a method of pulling it from
>> > them (or them pushing it out to you), unless you're in the unique
>> > position to dictate the decision for them.  
>> 
>> They need to use ftp to deliver the data, we need to use ftp to get
>> the data.  I don't want that any other way.
>> 
>> The problem is that the ones supposed to deliver data are incompetent
>> and don't want to use ftp because it's too complicated.  So what's the
>> better solution?
>
> Use an edge router appliance with proper VPN support.

That's what I'm doing, and it doesn't make VPN easy.  I guess that lies
in the nature of VPN.

> You are from Germany? I can recommend Securepoint appliances. You pay
> for the hardware and support, they support you with setting everything
> up. You can also find a distributor who can install this for
> you. Securepoint works with competent partners all around Germany.

That would probably cost a lot of money, and external support always
involves significant delays.  I'll just have to learn it myself.

> There's also other alternatives like Watchguard (but their OpenVPN
> support is not that good), and a lot of free router/firewall softwares
> you can deploy to semi-professional equipment by firmware replacement.
> But at least with the latter option, you're mostly on your own and need
> to invest a lot of effort to make it work properly and secure.

Yes, that would make it much more complicated than it needs to be.

> Depending on what data is transferred, you should also take into
> account if your solution is certificated to transfer such data. E.g.
> medical data may only be transferred through properly certificated VPN
> appliances. Otherwise, you should fall back to sneakernet. I'm not sure
> how that is any more secure but that's how things are.

Interesting, who certifies such appliances?  What if I, as a patient, do
not want my data transferred that way, and how do I know if they didn't
make a mistake when certifying the equipment?

It's not medical data, and nobody in Germany actually cares about
protecting peoples data anyway.  The little that is being done towards
that is nothing but pretense.


-- 
"Didn't work" is an error.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sun, 14 May 2017 01:28:55 +0100
> schrieb lee :
>
>> Kai Krakow  writes:
>> 
>> > Am Sat, 29 Apr 2017 22:02:51 -0400
>> > schrieb "Walter Dnes" :
>> >  
>> >>   Then there's always "sneakernet".  To quote Andrew Tanenbaum from
>> >> 1981
>> >>   
>>  [...]  
>> >
>> > Hehe, with the improvements in internet connections nowadays, we
>> > almost stopped transferring backups via sneakernet. Calculating the
>> > transfer speed of the internet connection vs. the speed calculating
>> > miles per hour, internet almost always won lately. :-)
>> >
>> > Most internet connections are faster than even USB sticks these
>> > days.  
>> 
>> Wow, you must be living in some sort of paradise.  Here, internet is
>> more like being cut off from the rest of the world.
>> 
>> But then, there's a manufacturer that makes incredibly slow USB sticks
>> which I won't buy anymore ...
>
> Okay, it really depends. I shouldn't say "most"... ;-)

Intenso --- pretty cheap, but awfully slow; however, it does
work. Better don't buy anything they make unless your time is entirely
worthless to you.

> I compared my really crappy (but most reliable yet) old USB stick to my
> internet connection. My USB stick doesn't do 48 MByte/s, more like 5-10.
> And don't even ask when writing data.

5--10MB/s?  How do you get that much?

> Even my rusty hard disk (read: not SSD) has a hard time writing away a
> big download with constantly high download rate.

It must be really old then, about 20 years.

> But I guess that a good internet connection should be at least 50 MBit
> these days.

I'd say 100, but see above.  The advantage is that you have sufficient
bandwidth to do several things at the same time.  I've never seen fast
internet.

> And most USB sticks are really crappy at writing. That also counts when
> you do not transfer the file via network. Of course, most DSL
> connections have crappy upload speed, too. Only lately, Telekom offers
> 40 MBit upload connections in Germany.

They offer 384kbit/s downstream and deliver 365.  It's almost
symmetrical, yet almost unusable.

They also offer 50Mbit and deliver between 2 and 12, and upstream is
awfully low.  Tell them you could pay for 16 instead of 50 because you
don't get even that much, and they will tell you that you would get even
less than you do now.  That is unacceptable.

And try to get a static IP so you could really use your connection ...

> I'm currently on a 400/25 MBit link and can saturate the link only with
> proper servers like the Steam network which can deliver 48 MByte/s.

You must be sitting in a data center and be very lucky to have that.


-- 
"Didn't work" is an error.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sun, 14 May 2017 02:18:56 +0100
> schrieb lee :
>
>> Kai Krakow  writes:
>> 
>> > Am Sat, 29 Apr 2017 20:02:57 +0100
>> > schrieb lee :
>> >  
>> >> Alan McKinnon  writes:
>> >>   
>>  [...]  
>>  [...]  
>>  [...]  
>> >> 
>> >> The intended users are incompetent, hence it is too difficult to
>> >> use ...  
>> >
>> > If you incompetent users are using Windows: Have you ever tried
>> > entering ftp://u...@yoursite.tld in the explorer directory input
>> > bar?  
>> 
>> I tried at work and it said something like that the service cannot be
>> accessed.
>> 
>> 
>> > [...]
>> > Debian is not the king to rule the internet. You shouldn't care when
>> > they shut down their FTP services. It doesn't matter to the rest of
>> > the world using the internet.  
>> 
>> Who can say what their influence actually is?  Imagine Debian going
>> away, and all the distributions depending on them as well because they
>> loose their packet sources, then what remains?  It is already rather
>> difficult to find a usable distribution, and what might the effect on
>> upstream sources be.
>
> The difference is: They only shut down a service. They are not
> vanishing from the internet. You cannot conclude from that, they are:
>
> (a) shutting down all their service
> (b) ftp is deprecated and nobody should use it any longer
>
> And I didn't write that you shouldn't care if Debian vanishes. I only
> said it shouldn't mean anything to you if they shut down their FTP
> services for probably good reasons. It's not the end of life, the
> universe, and everything. And you can keep your towel.
>
> What I wanted to say: Debian is not that important that everyone will
> shut down FTP now and kill FTP support from client software. That
> simply won't happen. That is not what it means when Debian is shutting
> down a service.

I didn't say that Debian is vanishing, only that I doubt that they are
without influence and that their influence might easily be
underestimated.


-- 
"Didn't work" is an error.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Mon, 15 May 2017 08:53:15 +0100
schrieb Mick :

> On Sunday 14 May 2017 11:35:29 Kai Krakow wrote:
> > Am Sun, 14 May 2017 09:52:41 +0100
> > 
> > schrieb Mick :  
> > > On Saturday 13 May 2017 23:58:17 R0b0t1 wrote:  
>  [...]  
> > > 
> > > OpenVPN is not the most efficient VPN implementation for
> > > connections to a server because it is not multithreaded  
> > 
> > Probably true but it works well here for connections of up to 100
> > MBit.  
> 
> It can work well for well above that throughput, but the limitation
> is the tun/tap mechanism and the CPU of the device/PC it is running
> on.

I think most important is to use the UDP transport and not TCP, because
the tunnel protocol doesn't need to ensure packet delivery. This is
done by the protocols running inside the tunnel. Also, we usually
enable compression at least for low-bandwidth uplinks (which become
rare these days fortunately).

To compensate for the UDP protocol, we usually also give the tunneling
packets higher priority at the edge router to reduce drop rate under
uplink pressure.

This works well for most dial-up links we encounter (currently up to
100 MBit). I probably won't consider it for higher throughput links
because I fear the appliance CPU may become a bottleneck. But so far,
no problems, even not with CPU usage.


> > > and also because unlike
> > > IKE/IPSec it operates in userspace, not in kernelspace.  
> > 
> > IPsec also doesn't work without help from userspace processes.   
> 
> Sure, but this is only for managing the (re)keying process, which BTW
> takes longer with IKE than with OpenVPN (we're talking about
> milliseconds here). Once the keys have been agreed and set up between
> peers the rest happens exceedingly fast in kernelspace, managed as a
> network layer interface (L3).  I recall seeing IPSec tunnels running
> 10 times faster than OpenVPN, being processed even faster than VLAN
> trunking, but this is very much dependent on the resources of the
> device running the tunnel.

I use IPsec only between to endpoints directly connected to the
internet (without NAT) and static IP. And only then it was really
reliable, and it performed well. No question about it...

And I like the fact that I don't need an intermediate transfer net as
opposed to OpenVPN.

OTOH, only OpenVPN has been reliable enough (and very reliable so far)
when one or both sides were NATed with dynamic IP.

And we had one customer running two networks across four sites, and
their IPsec solution never ran reliable. And this was with
professional, expensive firewall appliances. We replaced it with
site-2-site OpenVPN and it runs faster now and without any disconnects.
All sites use static IPs so that was never the problem. I don't know
what caused this. The old appliances were mostly blackboxes, and at
least one was faulty hardware (which explained problems at one site).


> > But I
> > see what you mean: With OpenVPN, traffic bounces between kernel and
> > userspace multiple times before leaving the machine. But I don't
> > really see that as a problem for the scenario OpenVPN is used in:
> > It best fits with dial-up connections which are really not gigabit
> > yet. For this, performance overhead is almost zero.  
> 
> Yes, at dial-up throughput even a smart phone has enough resources to
> manage OpenVPN without it becoming a constraint.
> 
> 
> > IPsec can be a big pita if NAT is involved. For Windows client, L2TP
> > may be a good alternative.  
> 
> IKE/IPSec uses NAT-Traversal (NAT-T) by encapsulating ESP packets
> within UDP over port 4500.  This will allow clients to initiate a
> connection with the server over port 500 and then switch to 4500 as
> part of NAT-T detection. Trivia:  many routers/VPN concentrators use
> Vendor ID strings to determine if the remote peer can implement NAT-T
> among other attributes to shorten this NAT-T detection process.
> 
> Of course the server will have to be accessible over port 500 for the
> clients to be able to get to it, but this is a port forwarding/DMZ
> network configuration exercise at the server end.

Oh wait... So I need to forward port 500 and 4500 so NAT-T does work
properly? Even when both sides are NATed? I never got that to work
reliably for one side NATed, and it never worked for both sides NATed.
And my research in support forums always said: That does not work...


>  [...]  
> > > 
> > > If the users are helpless then you may be better configuring a VPN
> > > tunnel between their Internet gateway and the server, so they can
> > > access the server as if it were a local share, or using the built
> > > in ftp client that MSWindows comes with.  SMB will work securely
> > > in this case too.  
> > 
> > This is what I would recommend, too. Put the VPN endpoints on the
> > network edges and no clients needs to worry: They just use the
> > connection.  
> 
> If there is a large number of client PCs this is the only sane
> solution.

There are 

Re: [gentoo-user] Re: replacement for ftp?

[Mick]

On Sunday 14 May 2017 11:35:29 Kai Krakow wrote:
> Am Sun, 14 May 2017 09:52:41 +0100
> 
> schrieb Mick :
> > On Saturday 13 May 2017 23:58:17 R0b0t1 wrote:
> > > I had some problems setting up OpenVPN that were solved by using
> > > per-client public keys. That seems to be the best supported
> > > configuration (as well as the most secure). Windows-side using
> > > OpenVPN-GUI is very easy.
> > > 
> > > OpenVPN tends to have poor bandwidth due to overhead, but that may
> > > be in large part due to my connection.
> > 
> > OpenVPN is not the most efficient VPN implementation for connections
> > to a server because it is not multithreaded
> 
> Probably true but it works well here for connections of up to 100 MBit.

It can work well for well above that throughput, but the limitation is the 
tun/tap mechanism and the CPU of the device/PC it is running on.


> > and also because unlike
> > IKE/IPSec it operates in userspace, not in kernelspace.
> 
> IPsec also doesn't work without help from userspace processes. 

Sure, but this is only for managing the (re)keying process, which BTW takes 
longer with IKE than with OpenVPN (we're talking about milliseconds here).  
Once the keys have been agreed and set up between peers the rest happens 
exceedingly fast in kernelspace, managed as a network layer interface (L3).  I 
recall seeing IPSec tunnels running 10 times faster than OpenVPN, being 
processed even faster than VLAN trunking, but this is very much dependent on 
the resources of the device running the tunnel.


> But I
> see what you mean: With OpenVPN, traffic bounces between kernel and
> userspace multiple times before leaving the machine. But I don't really
> see that as a problem for the scenario OpenVPN is used in: It best fits
> with dial-up connections which are really not gigabit yet. For this,
> performance overhead is almost zero.

Yes, at dial-up throughput even a smart phone has enough resources to manage 
OpenVPN without it becoming a constraint.


> IPsec can be a big pita if NAT is involved. For Windows client, L2TP
> may be a good alternative.

IKE/IPSec uses NAT-Traversal (NAT-T) by encapsulating ESP packets within UDP 
over port 4500.  This will allow clients to initiate a connection with the 
server over port 500 and then switch to 4500 as part of NAT-T detection.  
Trivia:  many routers/VPN concentrators use Vendor ID strings to determine if 
the remote peer can implement NAT-T among other attributes to shorten this 
NAT-T detection process.

Of course the server will have to be accessible over port 500 for the clients 
to be able to get to it, but this is a port forwarding/DMZ network 
configuration exercise at the server end.


> > > > The ftp server already doesn't allow unencrypted connections.
> > > > 
> > > > Now try to explain to ppl for whom Filezilla is too complicated
> > > > how to set up a VPN connection and how to secure their LAN once
> > > > they create the connection (if we could ever get that to work).
> > > > I haven't been able to figure that out myself, and that is one of
> > > > the main reasons why I do not have a VPN connection but use ssh
> > > > instead.  The only disadvantage is that I can't do RDP sessions
> > > > with that ---  I probably could and just don't know how to ---
> > > > but things might be a lot easier if wireguard works.
> > 
> > If the users are helpless then you may be better configuring a VPN
> > tunnel between their Internet gateway and the server, so they can
> > access the server as if it were a local share, or using the built in
> > ftp client that MSWindows comes with.  SMB will work securely in this
> > case too.
> 
> This is what I would recommend, too. Put the VPN endpoints on the
> network edges and no clients needs to worry: They just use the
> connection.

If there is a large number of client PCs this is the only sane solution.


> >  [...]
> >  
> > > > I'm finding it a horrible nightmare, see above.  It is the most
> > > > difficult thing you could come up with.  I haven't found any good
> > > > documentation that explains it, the different types of it, how it
> > > > works, what to use (apparently there are many different ways or
> > > > something, some of which require a static IP on both ends, and
> > > > they even give you different disadvantages in performance ...),
> > > > how to protect the participants and all the complicated stuff
> > > > involved.  So far, I've managed to stay away from it, and I
> > > > wouldn't know where to start.  Of course, there is some
> > > > documentation, but it is all confusing and no good.
> > > 
> > > Feel free to start a thread on it. As above, I recommend
> > > one-key-per-client and running your own CA.
> 
> I wouldn't recommend running your own CA because you will have to
> deploy a trust relationship with every client.
> 
> > For secure connections you will have to set up CA and TLS keys with
> > any option.  Even ftps - unless the ftp server is already 

Re: [gentoo-user] Re: replacement for ftp?

[Kent Fredric]

On Sun, 14 May 2017 02:59:41 +0100
lee  wrote:

> That requires shell access.

Not necessarily, it just requires a competent ISP.

For instance, there's no shell access on github, but there's still
ssh-based sync.

So you just need to have a restricted environment that only allows
spawning of the server side parts of the sftp protocol, and a suitable
authentication scheme.

You're going to want authentication for push anyway.


pgpcRbWjSdB36.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: replacement for ftp?

[R0b0t1]

On Sun, May 14, 2017 at 4:47 PM, R0b0t1  wrote:
> That is only in one setup. It is possible to assign an IP address to
> OpenVPN such that you will need any traffic to cross onto your LAN.
>

Whoops: "It is possible to assign an IP address to OpenVPN such that
you will need routing rules for any traffic to cross onto your LAN."

Re: [gentoo-user] Re: replacement for ftp?

[R0b0t1]

On Sun, May 14, 2017 at 3:52 AM, Mick  wrote:
>> I had some problems setting up OpenVPN that were solved by using
>> per-client public keys. That seems to be the best supported
>> configuration (as well as the most secure). Windows-side using
>> OpenVPN-GUI is very easy.
>>
>> OpenVPN tends to have poor bandwidth due to overhead, but that may be
>> in large part due to my connection.
>
> OpenVPN is not the most efficient VPN implementation for connections to a
> server because it is not multithreaded and also because unlike IKE/IPSec it
> operates in userspace, not in kernelspace.  If you have more than one client
> connecting to the server at the same time you will need to set up multiple
> instances with different ports or different protocols.  With IKE/IPSec you
> don't.  MSWindows PCs come with IKEv2 natively so they can be configured to
> use it without installing additional client applications.
>
> A VPN connection will expose each endpoint's LAN to the other and therefore
> additional firewall configurations could be required.
>

That is only in one setup. It is possible to assign an IP address to
OpenVPN such that you will need any traffic to cross onto your LAN.

>> >> OpenVPN also offers transparent compression which can be a big
>> >> plus for your scenario.
>> >
>> > Not really, a lot of data is images, usually JPEG, some ZIP files, some
>> > PDF.  All that doesn't compress too well.
>> >
>> >> OpenVPN is not too difficult to setup, and the client is available for
>> >> all major OSes. And it's not too complicated to use: Open VPN
>> >> connection, then use your file transfer client as you're used to. Just
>> >> one simple extra step.
>> >
>> > I'm finding it a horrible nightmare, see above.  It is the most
>> > difficult thing you could come up with.  I haven't found any good
>> > documentation that explains it, the different types of it, how it works,
>> > what to use (apparently there are many different ways or something, some
>> > of which require a static IP on both ends, and they even give you
>> > different disadvantages in performance ...), how to protect the
>> > participants and all the complicated stuff involved.  So far, I've
>> > managed to stay away from it, and I wouldn't know where to start.  Of
>> > course, there is some documentation, but it is all confusing and no
>> > good.
>>
>> Feel free to start a thread on it. As above, I recommend
>> one-key-per-client and running your own CA.
>
> For secure connections you will have to set up CA and TLS keys with any
> option.  Even ftps - unless the ftp server is already configured with its TLS
> certificates.
>

No, certain OpenVPN modes allow encryption without a CA, but they are
limited (e.g. single user, single password, etc).

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sun, 14 May 2017 01:25:24 +0100
schrieb lee :

> "Poison BL."  writes:
> 
> > On Sat, Apr 29, 2017 at 9:11 PM, lee  wrote:  
> >>
> >> "Poison BL."  writes:  
>  [...]  
> > trust  
>  [...]  
> >>
> >> Why not?  (12GB are nowhere close to half a petabyte ...)  
> >
> > Ah... I completely misread that "or over 50k files in 12GB" as 50k
> > files *at* 12GB each... which works out to 0.6 PB, incidentally.
> >  
> >> The data would come in from suppliers.  There isn't really anything
> >> going on atm but fetching data once a month which can be like
> >> 100MB or 12GB or more.  That's because ppl don't use ftp ...  
> >
> > Really, if you're pulling it in from third party suppliers, you
> > tend to be tied to what they offer as a method of pulling it from
> > them (or them pushing it out to you), unless you're in the unique
> > position to dictate the decision for them.  
> 
> They need to use ftp to deliver the data, we need to use ftp to get
> the data.  I don't want that any other way.
> 
> The problem is that the ones supposed to deliver data are incompetent
> and don't want to use ftp because it's too complicated.  So what's the
> better solution?

Use an edge router appliance with proper VPN support. You are from
Germany? I can recommend Securepoint appliances. You pay for the
hardware and support, they support you with setting everything up. You
can also find a distributor who can install this for you. Securepoint
works with competent partners all around Germany.

There's also other alternatives like Watchguard (but their OpenVPN
support is not that good), and a lot of free router/firewall softwares
you can deploy to semi-professional equipment by firmware replacement.
But at least with the latter option, you're mostly on your own and need
to invest a lot of effort to make it work properly and secure.

Depending on what data is transferred, you should also take into
account if your solution is certificated to transfer such data. E.g.
medical data may only be transferred through properly certificated VPN
appliances. Otherwise, you should fall back to sneakernet. I'm not sure
how that is any more secure but that's how things are.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sun, 14 May 2017 01:28:55 +0100
schrieb lee :

> Kai Krakow  writes:
> 
> > Am Sat, 29 Apr 2017 22:02:51 -0400
> > schrieb "Walter Dnes" :
> >  
> >>   Then there's always "sneakernet".  To quote Andrew Tanenbaum from
> >> 1981
> >>   
>  [...]  
> >
> > Hehe, with the improvements in internet connections nowadays, we
> > almost stopped transferring backups via sneakernet. Calculating the
> > transfer speed of the internet connection vs. the speed calculating
> > miles per hour, internet almost always won lately. :-)
> >
> > Most internet connections are faster than even USB sticks these
> > days.  
> 
> Wow, you must be living in some sort of paradise.  Here, internet is
> more like being cut off from the rest of the world.
> 
> But then, there's a manufacturer that makes incredibly slow USB sticks
> which I won't buy anymore ...

Okay, it really depends. I shouldn't say "most"... ;-)

I compared my really crappy (but most reliable yet) old USB stick to my
internet connection. My USB stick doesn't do 48 MByte/s, more like 5-10.
And don't even ask when writing data.

Even my rusty hard disk (read: not SSD) has a hard time writing away a
big download with constantly high download rate.

But I guess that a good internet connection should be at least 50 MBit
these days.

And most USB sticks are really crappy at writing. That also counts when
you do not transfer the file via network. Of course, most DSL
connections have crappy upload speed, too. Only lately, Telekom offers
40 MBit upload connections in Germany.

I'm currently on a 400/25 MBit link and can saturate the link only with
proper servers like the Steam network which can deliver 48 MByte/s.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sun, 14 May 2017 02:18:56 +0100
schrieb lee :

> Kai Krakow  writes:
> 
> > Am Sat, 29 Apr 2017 20:02:57 +0100
> > schrieb lee :
> >  
> >> Alan McKinnon  writes:
> >>   
>  [...]  
>  [...]  
>  [...]  
> >> 
> >> The intended users are incompetent, hence it is too difficult to
> >> use ...  
> >
> > If you incompetent users are using Windows: Have you ever tried
> > entering ftp://u...@yoursite.tld in the explorer directory input
> > bar?  
> 
> I tried at work and it said something like that the service cannot be
> accessed.
> 
> 
> > [...]
> > Debian is not the king to rule the internet. You shouldn't care when
> > they shut down their FTP services. It doesn't matter to the rest of
> > the world using the internet.  
> 
> Who can say what their influence actually is?  Imagine Debian going
> away, and all the distributions depending on them as well because they
> loose their packet sources, then what remains?  It is already rather
> difficult to find a usable distribution, and what might the effect on
> upstream sources be.

The difference is: They only shut down a service. They are not
vanishing from the internet. You cannot conclude from that, they are:

(a) shutting down all their service
(b) ftp is deprecated and nobody should use it any longer

And I didn't write that you shouldn't care if Debian vanishes. I only
said it shouldn't mean anything to you if they shut down their FTP
services for probably good reasons. It's not the end of life, the
universe, and everything. And you can keep your towel.

What I wanted to say: Debian is not that important that everyone will
shut down FTP now and kill FTP support from client software. That
simply won't happen. That is not what it means when Debian is shutting
down a service.


-- 
Regards,
Kai

Replies to list-only preferred.

Re: [gentoo-user] Re: replacement for ftp?

[Neil Bothwick]

On Sun, 14 May 2017 02:48:46 +0100, lee wrote:

> > But you could offer access via OpenVPN and tunnel samba through
> > that.  
> 
> I haven't been able yet to figure out what implications creating a VPN
> has.  I understand it's supposed to connect networks through a secured
> tunnel, but what kind of access to the LAN does someone get who connects
> via VPN?  Besides, VPN is extremely complicated and difficult to set
> up.  I consider it an awful nightmare.

Try ZeroTier, it's easy to install and setup (there's an ebuild on b.g.o)
and provides VPN like access, but to individual machines on your network
instead of exposing the entire network to these incompetent people. You
administer ZeroTier centrally, the users only need to install the
software and join the network - then you can give them access to the
Samba server and nothing else on your network.

However, I think owncloud/nextcloud might be a better option. As long as
your users can use a web browser, they can use this and no extra software
is needed.

Another option may be Syncthing, which syncs a local directory with one
over the network. It's encrypted and P2P and needs minimal setup on the
client side. Once set up, they simply copy the files to a directory,
which is synced in the background.


-- 
Neil Bothwick

Hickory Dickory Dock, The mice ran up the clock, The clock struck one, The
others escaped with minor injuries.


pgpSsem40BZXE.pgp
Description: OpenPGP digital signature

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sun, 14 May 2017 09:52:41 +0100
schrieb Mick :

> On Saturday 13 May 2017 23:58:17 R0b0t1 wrote:
> > I had some problems setting up OpenVPN that were solved by using
> > per-client public keys. That seems to be the best supported
> > configuration (as well as the most secure). Windows-side using
> > OpenVPN-GUI is very easy.
> > 
> > OpenVPN tends to have poor bandwidth due to overhead, but that may
> > be in large part due to my connection.  
> 
> OpenVPN is not the most efficient VPN implementation for connections
> to a server because it is not multithreaded

Probably true but it works well here for connections of up to 100 MBit.

> and also because unlike
> IKE/IPSec it operates in userspace, not in kernelspace.

IPsec also doesn't work without help from userspace processes. But I
see what you mean: With OpenVPN, traffic bounces between kernel and
userspace multiple times before leaving the machine. But I don't really
see that as a problem for the scenario OpenVPN is used in: It best fits
with dial-up connections which are really not gigabit yet. For this,
performance overhead is almost zero.


>  If you have
> more than one client connecting to the server at the same time you
> will need to set up multiple instances with different ports or
> different protocols.

That is not true: We connect many clients to the same server port
without problems, each with their own certificate.

>  With IKE/IPSec you don't.  MSWindows PCs come
> with IKEv2 natively so they can be configured to use it without
> installing additional client applications.

IPsec can be a big pita if NAT is involved. For Windows client, L2TP
may be a good alternative.

>  [...]  
> > > 
> > > The ftp server already doesn't allow unencrypted connections.
> > > 
> > > Now try to explain to ppl for whom Filezilla is too complicated
> > > how to set up a VPN connection and how to secure their LAN once
> > > they create the connection (if we could ever get that to work).
> > > I haven't been able to figure that out myself, and that is one of
> > > the main reasons why I do not have a VPN connection but use ssh
> > > instead.  The only disadvantage is that I can't do RDP sessions
> > > with that ---  I probably could and just don't know how to ---
> > > but things might be a lot easier if wireguard works.  
> 
> If the users are helpless then you may be better configuring a VPN
> tunnel between their Internet gateway and the server, so they can
> access the server as if it were a local share, or using the built in
> ftp client that MSWindows comes with.  SMB will work securely in this
> case too.

This is what I would recommend, too. Put the VPN endpoints on the
network edges and no clients needs to worry: They just use the
connection.

>  [...]  
> > > 
> > > I'm finding it a horrible nightmare, see above.  It is the most
> > > difficult thing you could come up with.  I haven't found any good
> > > documentation that explains it, the different types of it, how it
> > > works, what to use (apparently there are many different ways or
> > > something, some of which require a static IP on both ends, and
> > > they even give you different disadvantages in performance ...),
> > > how to protect the participants and all the complicated stuff
> > > involved.  So far, I've managed to stay away from it, and I
> > > wouldn't know where to start.  Of course, there is some
> > > documentation, but it is all confusing and no good.  
> > 
> > Feel free to start a thread on it. As above, I recommend
> > one-key-per-client and running your own CA.  

I wouldn't recommend running your own CA because you will have to
deploy a trust relationship with every client.

> For secure connections you will have to set up CA and TLS keys with
> any option.  Even ftps - unless the ftp server is already configured
> with its TLS certificates.

Or you use certificates from LetsEncrypt. Their CA is already trusted
on most machines my default.


-- 
Regards,
Kai

Replies to list-only preferred.


pgpoaF9L6wIsl.pgp
Description: Digitale Signatur von OpenPGP

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sun, 14 May 2017 02:48:46 +0100
schrieb lee :

> Kai Krakow  writes:
> 
> > Am Sat, 29 Apr 2017 20:30:03 +0100
> > schrieb lee :
> >  
> >> Danny YUE  writes:
> >>   
>  [...]  
>  [...]  
>  [...]  
> >> 
> >> Doesn't that require ssh access?  And how do you explain that to
> >> ppl finding it too difficult to use Filezilla?  Is it available for
> >> Windoze?  
> >
> > Both, sshfs and scp, require a full shell (that may be restricted
> > but that involves configuration overhead on the server side).  
> 
> I wouldn't want them to have that.

And I can understand this...

> > You can use sftp (FTP wrapped into SSH), which is built into SSH. It
> > has native support in many Windows clients (most implementations use
> > PuTTY in the background). It also has the advantage that you can
> > easily restrict users on your system to SFTP-only with an easy
> > server-side configuration.  
> 
> From what I've been reading, sftp is deprecated and has been replaced
> by ftp with TLS.

From what I'm guessing, you're mixing up sftp and ftps. sftp is
ssh+ftp, and ftps is ftp with ssl. The latter is probably deprecated in
favor of ftp with tls. TLS supports name indication (to show the
correct server certificate) and it supports handshaking so the same
port can be used for secure and insecure connections.

Apparently, many sites on the internet also mix up ftps und sftp, for
them both is FTP with SSL. But that's not true. I think that comes from
the fact that "secure ftp" often is a synonym for "ssl encryption" as
it is with "secure http". But that doesn't mean the acronym is "sftp"
as it also is not "shttp".

>  [...]  
> >> 
> >> Does that work well, reliably and securely over internet
> >> connections?  
> >
> > It supports encryption as transport security, and it supports
> > kerberos for secure authentication, the latter is not easy to setup
> > in Linux, but it should work with Windows clients out-of-the-box.
> >
> > But samba is a pretty complex daemon and thus offers a big attack
> > surface for hackers and bots. I'm not sure you want to expose this
> > to the internet without some sort of firewall in place to restrict
> > access to specific clients - and that probably wouldn't work for
> > your scenario.  
> 
> At least it's a possibility.  I don't even know if they have static
> IPs, though.

Modern CIFS implementations can be forced to encrypt the transport
layer and only accept kerberos authenticated clients. It should be safe
to use then if properly firewalled. At least "CIFS" (which is samba)
afaik means "common internet file system" - that should at least have a
minimal meaning of "intended to be used over internet connections". Of
course this really doesn't say anything about transport security. Be
sure to apply one, and you should be good to go.

> > But you could offer access via OpenVPN and tunnel samba through
> > that.  
> 
> I haven't been able yet to figure out what implications creating a VPN
> has.  I understand it's supposed to connect networks through a secured
> tunnel, but what kind of access to the LAN does someone get who
> connects via VPN?  Besides, VPN is extremely complicated and
> difficult to set up.  I consider it an awful nightmare.

You need to first understand how tunnel devices work. Then it becomes
very easy to set up. The access to the LAN can be restricted by
firewall rules. As long as you don't setup routes from the transfer
network (where the tunnel is located) to your LAN, there won't be
access. And then there's firewall rules after you set up routing.

> Wireguard seems a lot easier.

I didn't know that, I will look into it.

> > By that time, you can as easily offer FTP, too, through the tunnel
> > only, as there should be no more security concerns now: It's
> > encrypted now.  
> 
> The ftp server already doesn't allow unencrypted connections.
> 
> Now try to explain to ppl for whom Filezilla is too complicated how to
> set up a VPN connection and how to secure their LAN once they create
> the connection (if we could ever get that to work).  I haven't been
> able to figure that out myself, and that is one of the main reasons
> why I do not have a VPN connection but use ssh instead.  The only
> disadvantage is that I can't do RDP sessions with that ---  I
> probably could and just don't know how to --- but things might be a
> lot easier if wireguard works.

You can always deploy VPN at the edge of the network, so your clients
won't need to bother with the details but just use the connection.

You can also try using WinSCP instead of filezilla (it supports,
despite the name, also FTP). Then put a connection file to their
desktop and configure it to run in explorer-mode. Now it should mostly
look like a file explorer and they can copy files like they used to.

But then again: Ppl want to get paid for their work. That also means
they need to invest at least a bit more than just their time... 

Re: [gentoo-user] Re: replacement for ftp?

[Adam Carter]

> > These certificates are a very stupid thing.  They are utterly
> > complicated, you have to self-sign them which produces warnings, and
> > they require to have the host name within them as if the host wasn't
> > known by several different names.
>
> Use LetsEncrypt then, you can add any number of host names you want, as
> far as I know. But you need a temporary web server to prove ownership
> of the server/hostname and sign the certificates.
>

As an alternative you can publish a DNS record, rather than providing a web
server.

I like https to distribute fires - its easiest for the users, and no client
end setup required.

If there are many files just zip them up as most users can handle that too,
and zip clients are build into most OSes.

Re: [gentoo-user] Re: replacement for ftp?

[Mick]

On Saturday 13 May 2017 23:58:17 R0b0t1 wrote:
> On Wed, May 3, 2017 at 1:40 AM, Mick  wrote:
> > On Monday 01 May 2017 22:36:00 Nils Freydank wrote:
> >> On Sat, 30 Apr 2017 19:04:06 +0200 Andrew Savchenko wrote:
> >> > [...]
> >> > I fail to see why FTP needs to be replaced: it works, it is
> >> > supported, it is secure when used with care, it is damn fast.
> >> 
> >> I’ll just drop the somewhat popular rant “FTP must die“[1] and a
> >> follow-up
> >> discussion about it[2]. IMHO the main reasons are missing data integrity
> >> and authentication security issues. The latter one can be solved with
> >> FTPS[3] - but honestly I never saw FTPS somewhere actually used in the
> >> wild.> 
> > I'm not sure what you mean "used in the wild".  I use lftp to connect via
> > ftps with a number of webservers for updates and backups on a daily
> > basis.  Some of the connections are scripted.
> > 
> >> [1] http://mywiki.wooledge.org/FtpMustDie
> >> [2] https://news.ycombinator.com/item?id=11251907
> >> [3] i.e. FTP over SSL/TLS (not to mix up with SFTP, which comes from the
> >> SSH family)
> >> 
> >> Greetings,
> >> Nils
> 
> That was an interesting read. The only RFC as bad as FTP's that I know
> of might be IRC's.
> 
> On Sat, May 13, 2017 at 8:48 PM, lee  wrote:
> > Kai Krakow  writes:
> >> Am Sat, 29 Apr 2017 20:30:03 +0100
> >> 
> >> schrieb lee :
> >>> Danny YUE  writes:
> >>> > On 2017-04-25 14:29, lee  wrote:
> >>> >> Hi,
> >>> >> 
> >>> >> since the usage of FTP seems to be declining, what is a replacement
> >>> >> which is at least as good as FTP?
> >>> >> 
> >>> >> I'm aware that there's webdav, but that's very awkward to use and
> >>> >> missing features.
> >>> > 
> >>> > What about sshfs? It allows you to mount a location that can be
> >>> > accessed via ssh to your local file system, as if you are using
> >>> > ssh.
> >>> 
> >>> Doesn't that require ssh access?  And how do you explain that to ppl
> >>> finding it too difficult to use Filezilla?  Is it available for
> >>> Windoze?
> >> 
> >> Both, sshfs and scp, require a full shell (that may be restricted but
> >> that involves configuration overhead on the server side).
> > 
> > I wouldn't want them to have that.
> > 
> >> You can use sftp (FTP wrapped into SSH), which is built into SSH. It
> >> has native support in many Windows clients (most implementations use
> >> PuTTY in the background). It also has the advantage that you can
> >> easily restrict users on your system to SFTP-only with an easy
> >> server-side configuration.
> > 
> > From what I've been reading, sftp is deprecated and has been replaced by
> > ftp with TLS.

MSWindows does not do ftps, only unencrypted ftp.  The solution is to use a 
MSWindows ftp client, life filezilla.  If the users can be trained, or have 
their PCs set up with Filezilla bookmarks for required connections to the ftp 
server, then this problem can be overcome.


> >>> > Also samba can be a replacement. I have a samba server on my OpenWRT
> >>> > router and use mount.cifs to mount it...
> >>> 
> >>> Does that work well, reliably and securely over internet connections?
> >> 
> >> It supports encryption as transport security, and it supports kerberos
> >> for secure authentication, the latter is not easy to setup in Linux,
> >> but it should work with Windows clients out-of-the-box.
> >> 
> >> But samba is a pretty complex daemon and thus offers a big attack
> >> surface for hackers and bots. I'm not sure you want to expose this to
> >> the internet without some sort of firewall in place to restrict access
> >> to specific clients - and that probably wouldn't work for your scenario.

SMB is being patched on a regular basis by MS to improve its security - the 
recent global Wannacry attack being a case in point.  I would think SMB is the 
most attacked protocol on a daily basis and trying to configure a SMB server 
from scratch, when an ftp server is already available would not be the wisest 
investment of time.


> > At least it's a possibility.  I don't even know if they have static IPs,
> > though.
> > 
> >> But you could offer access via OpenVPN and tunnel samba through that.
> > 
> > I haven't been able yet to figure out what implications creating a VPN
> > has.  I understand it's supposed to connect networks through a secured
> > tunnel, but what kind of access to the LAN does someone get who connects
> > via VPN?  Besides, VPN is extremely complicated and difficult to set
> > up.  I consider it an awful nightmare.
> > 
> > Wireguard seems a lot easier.

I hadn't heard of its dev, Jason A. Donenfeld, who has trademarked the 
Wireguard name and website, until I visited the Wireguard website.  I don't 
know how many people are part of the team, but Jason appears to be the only 
person answering questions on the M/L.  The website advises that wireguard is 
work in progress and therefore 

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sun, 14 May 2017 02:59:41 +0100
schrieb lee :

> Kai Krakow  writes:
> 
> > Am Sat, 29 Apr 2017 20:38:24 +0100
> > schrieb lee :
> >  
> >> Kai Krakow  writes:
> >>   
>  [...]  
>  [...]  
>  [...]  
> >> 
> >> Yes, I'm using it mostly for backups/copies.
> >> 
> >> The problem is that ftp is ideal for the purpose, yet users find it
> >> too difficult to use, and nobody uses it.  So there must be
> >> something else as good or better which is easier to use and which
> >> ppl do use.  
> >
> > Well, I don't see how FTP is declining, except that it is
> > unencrypted. You can still use FTP with TLS handshaking, most sites
> > should support it these days but almost none forces correct
> > certificates because it is usually implemented wrong on the server
> > side (by giving you ftp.yourdomain.tld as the hostname instead of
> > ftp.hostingprovider.tld which the TLS cert has been issued for).
> > That makes it rather pointless to use. In linux, lftp is one of the
> > few FTP clients supporting TLS out-of-the-box by default, plus it
> > forces correct certificates.  
> 
> These certificates are a very stupid thing.  They are utterly
> complicated, you have to self-sign them which produces warnings, and
> they require to have the host name within them as if the host wasn't
> known by several different names.

Use LetsEncrypt then, you can add any number of host names you want, as
far as I know. But you need a temporary web server to prove ownership
of the server/hostname and sign the certificates.

> > But I found FTP being extra slow on small files, that's why I
> > suggested to use rsync instead. That means, where you could use
> > sftp (ssh+ftp), you can usually also use ssh+rsync which is
> > faster.  
> 
> That requires shell access.
> 
> What do you consider "small files"?  I haven't observed a slowdown
> like that, but I haven't been looking for it, either.

Transfer 1 smallish files (like web assets, php files) to a server
with FTP, then try rsync. You should see a very big difference in time
needed. That's due to the connection overhead of FTP.

> > There's also the mirror command in lftp, which can be pretty fast,
> > too, on incremental updates but still much slower than rsync.
> >  
> >> I don't see how they would transfer files without ftp when ftp is
> >> the ideal solution.  
> >
> > You simply don't. FTP is still there and used. If you see something
> > like "sftp" (ssh+ftp, not ftp+ssl which I would refer to as ftps),
> > this is usually only ftp wrapped into ssh for security reasons. It
> > just using ftp through a tunnel, but to the core it's the ftp
> > protocol. In the end, it's not much different to scp, as ftp is
> > really just only a special shell with some special commands to
> > setup a file transfer channel that's not prone to interact with
> > terminal escape sequences in whatever way those may be implemented,
> > something that e.g. rzsz needs to work around.
> >
> > In the early BBS days, where you couldn't establish a second
> > transfer channel like FTP does it using TCP, you had to send
> > special escape sequences to put the terminal into file transfer
> > mode, and then send the file. By that time, you used rzsz from the
> > remote shell to initiate a file transfer. This is more the idea of
> > how scp implements a file transfer behind the scenes.  
> 
> IIRC, I used xmodem or something like that back then, and rzsz never
> worked.

Yes, or xmodem... ;-)

> > FTP also added some nice features like site-to-site transfers where
> > the data endpoints both are on remote sites, and your local site
> > only is the control channel. This directly transfers data from one
> > remote site to another without going through your local connection
> > (which may be slow due to the dial-up nature of most customer
> > internet connections).  
> 
> Interesting, I didn't know that.  How do you do that?

You need a client that supports this. I remember LeechFTP for Windows
supported it back then. The client needs to log in to both FTP servers
and then shuffle correct PORT commands between them, so that the data
connection is directly established between both.

That feature is also the reason why this looks so overly complicated
and incompatible to firewalls. When FTP was designed, there was a real
need to directly transfer files between servers as your connection was
usually a slow modem connection below 2400 baud, or some other slow
connection. Or even one that wouldn't transfer binary data at all...

> > Also, FTP is able to stream multiple files in a single connection
> > for transferring many small files, by using tar as the transport
> > protocol, thus reducing the overhead of establishing a new
> > connection per file. Apparently, I know only few clients that
> > support that, and even fewer servers which that would with.
> >
> > FTP can be pretty powerful, as you see. It's just victim of its poor
> > 

Re: [gentoo-user] Re: replacement for ftp?

[R0b0t1]

On Wed, May 3, 2017 at 1:40 AM, Mick  wrote:
> On Monday 01 May 2017 22:36:00 Nils Freydank wrote:
>> On Sat, 30 Apr 2017 19:04:06 +0200 Andrew Savchenko wrote:
>> > [...]
>> > I fail to see why FTP needs to be replaced: it works, it is
>> > supported, it is secure when used with care, it is damn fast.
>>
>> I’ll just drop the somewhat popular rant “FTP must die“[1] and a follow-up
>> discussion about it[2]. IMHO the main reasons are missing data integrity and
>> authentication security issues. The latter one can be solved with FTPS[3] -
>> but honestly I never saw FTPS somewhere actually used in the wild.
>
> I'm not sure what you mean "used in the wild".  I use lftp to connect via ftps
> with a number of webservers for updates and backups on a daily basis.  Some of
> the connections are scripted.
>
>
>> [1] http://mywiki.wooledge.org/FtpMustDie
>> [2] https://news.ycombinator.com/item?id=11251907
>> [3] i.e. FTP over SSL/TLS (not to mix up with SFTP, which comes from the SSH
>> family)
>>
>> Greetings,
>> Nils
>

That was an interesting read. The only RFC as bad as FTP's that I know
of might be IRC's.


On Sat, May 13, 2017 at 8:48 PM, lee  wrote:
> Kai Krakow  writes:
>
>> Am Sat, 29 Apr 2017 20:30:03 +0100
>> schrieb lee :
>>
>>> Danny YUE  writes:
>>>
>>> > On 2017-04-25 14:29, lee  wrote:
>>> >> Hi,
>>> >>
>>> >> since the usage of FTP seems to be declining, what is a replacement
>>> >> which is at least as good as FTP?
>>> >>
>>> >> I'm aware that there's webdav, but that's very awkward to use and
>>> >> missing features.
>>> >
>>> > What about sshfs? It allows you to mount a location that can be
>>> > accessed via ssh to your local file system, as if you are using
>>> > ssh.
>>>
>>> Doesn't that require ssh access?  And how do you explain that to ppl
>>> finding it too difficult to use Filezilla?  Is it available for
>>> Windoze?
>>
>> Both, sshfs and scp, require a full shell (that may be restricted but
>> that involves configuration overhead on the server side).
>
> I wouldn't want them to have that.
>
>> You can use sftp (FTP wrapped into SSH), which is built into SSH. It
>> has native support in many Windows clients (most implementations use
>> PuTTY in the background). It also has the advantage that you can
>> easily restrict users on your system to SFTP-only with an easy
>> server-side configuration.
>
> From what I've been reading, sftp is deprecated and has been replaced by
> ftp with TLS.
>
>>> > Also samba can be a replacement. I have a samba server on my OpenWRT
>>> > router and use mount.cifs to mount it...
>>>
>>> Does that work well, reliably and securely over internet connections?
>>
>> It supports encryption as transport security, and it supports kerberos
>> for secure authentication, the latter is not easy to setup in Linux,
>> but it should work with Windows clients out-of-the-box.
>>
>> But samba is a pretty complex daemon and thus offers a big attack
>> surface for hackers and bots. I'm not sure you want to expose this to
>> the internet without some sort of firewall in place to restrict access
>> to specific clients - and that probably wouldn't work for your scenario.
>
> At least it's a possibility.  I don't even know if they have static IPs,
> though.
>
>> But you could offer access via OpenVPN and tunnel samba through that.
>
> I haven't been able yet to figure out what implications creating a VPN
> has.  I understand it's supposed to connect networks through a secured
> tunnel, but what kind of access to the LAN does someone get who connects
> via VPN?  Besides, VPN is extremely complicated and difficult to set
> up.  I consider it an awful nightmare.
>
> Wireguard seems a lot easier.
>

I had some problems setting up OpenVPN that were solved by using
per-client public keys. That seems to be the best supported
configuration (as well as the most secure). Windows-side using
OpenVPN-GUI is very easy.

OpenVPN tends to have poor bandwidth due to overhead, but that may be
in large part due to my connection.

>> By that time, you can as easily offer FTP, too, through the tunnel
>> only, as there should be no more security concerns now: It's encrypted
>> now.
>
> The ftp server already doesn't allow unencrypted connections.
>
> Now try to explain to ppl for whom Filezilla is too complicated how to
> set up a VPN connection and how to secure their LAN once they create the
> connection (if we could ever get that to work).  I haven't been able to
> figure that out myself, and that is one of the main reasons why I do not
> have a VPN connection but use ssh instead.  The only disadvantage is
> that I can't do RDP sessions with that ---  I probably could and just
> don't know how to --- but things might be a lot easier if wireguard
> works.
>
>> OpenVPN also offers transparent compression which can be a big
>> plus for your scenario.
>
> Not really, a lot of data 

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sat, 29 Apr 2017 20:38:24 +0100
> schrieb lee :
>
>> Kai Krakow  writes:
>> 
>> > Am Tue, 25 Apr 2017 15:29:18 +0100
>> > schrieb lee :
>> >  
>> >> since the usage of FTP seems to be declining, what is a replacement
>> >> which is at least as good as FTP?
>> >> 
>> >> I'm aware that there's webdav, but that's very awkward to use and
>> >> missing features.  
>> >
>> > If you want to sync files between two sites, try rsync. It is
>> > supported through ssh also. Plus, it's very fast also.  
>> 
>> Yes, I'm using it mostly for backups/copies.
>> 
>> The problem is that ftp is ideal for the purpose, yet users find it
>> too difficult to use, and nobody uses it.  So there must be something
>> else as good or better which is easier to use and which ppl do use.
>
> Well, I don't see how FTP is declining, except that it is unencrypted.
> You can still use FTP with TLS handshaking, most sites should support
> it these days but almost none forces correct certificates because it is
> usually implemented wrong on the server side (by giving you
> ftp.yourdomain.tld as the hostname instead of ftp.hostingprovider.tld
> which the TLS cert has been issued for). That makes it rather pointless
> to use. In linux, lftp is one of the few FTP clients supporting TLS
> out-of-the-box by default, plus it forces correct certificates.

These certificates are a very stupid thing.  They are utterly
complicated, you have to self-sign them which produces warnings, and
they require to have the host name within them as if the host wasn't
known by several different names.

> But I found FTP being extra slow on small files, that's why I suggested
> to use rsync instead. That means, where you could use sftp (ssh+ftp),
> you can usually also use ssh+rsync which is faster.

That requires shell access.

What do you consider "small files"?  I haven't observed a slowdown like
that, but I haven't been looking for it, either.

> There's also the mirror command in lftp, which can be pretty fast, too,
> on incremental updates but still much slower than rsync.
>
>> I don't see how they would transfer files without ftp when ftp is the
>> ideal solution.
>
> You simply don't. FTP is still there and used. If you see something
> like "sftp" (ssh+ftp, not ftp+ssl which I would refer to as ftps), this
> is usually only ftp wrapped into ssh for security reasons. It just
> using ftp through a tunnel, but to the core it's the ftp protocol. In
> the end, it's not much different to scp, as ftp is really just only a
> special shell with some special commands to setup a file transfer
> channel that's not prone to interact with terminal escape sequences in
> whatever way those may be implemented, something that e.g. rzsz needs
> to work around.
>
> In the early BBS days, where you couldn't establish a second transfer
> channel like FTP does it using TCP, you had to send special escape
> sequences to put the terminal into file transfer mode, and then send
> the file. By that time, you used rzsz from the remote shell to initiate
> a file transfer. This is more the idea of how scp implements a file
> transfer behind the scenes.

IIRC, I used xmodem or something like that back then, and rzsz never
worked.

> FTP also added some nice features like site-to-site transfers where the
> data endpoints both are on remote sites, and your local site only is
> the control channel. This directly transfers data from one remote site
> to another without going through your local connection (which may be
> slow due to the dial-up nature of most customer internet connections).

Interesting, I didn't know that.  How do you do that?

> Also, FTP is able to stream multiple files in a single connection for
> transferring many small files, by using tar as the transport protocol,
> thus reducing the overhead of establishing a new connection per file.
> Apparently, I know only few clients that support that, and even fewer
> servers which that would with.
>
> FTP can be pretty powerful, as you see. It's just victim of its poor
> implementation in most FTP clients that makes you feel it's mostly
> declined. If wrapped into a more secure tunnel (TLS, ssh), FTP is still
> a very good choice for transferring files, tho not the most efficient.
> Depending on your use case, you get away much better using more
> efficient protocols like rsync.

So there isn't a better solution than ftp.  That's good to know because
I can say there isn't a better solution, and if ppl don't want to use
it, they can send emails or DVDs.


-- 
"Didn't work" is an error.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sat, 29 Apr 2017 22:02:51 -0400
> schrieb "Walter Dnes" :
>
>>   Then there's always "sneakernet".  To quote Andrew Tanenbaum from
>> 1981
>> 
>> > Never underestimate the bandwidth of a station wagon full of tapes
>> > hurtling down the highway.  
>
> Hehe, with the improvements in internet connections nowadays, we
> almost stopped transferring backups via sneakernet. Calculating the
> transfer speed of the internet connection vs. the speed calculating
> miles per hour, internet almost always won lately. :-)
>
> Most internet connections are faster than even USB sticks these days.

Wow, you must be living in some sort of paradise.  Here, internet is
more like being cut off from the rest of the world.

But then, there's a manufacturer that makes incredibly slow USB sticks
which I won't buy anymore ...


-- 
"Didn't work" is an error.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sat, 29 Apr 2017 20:30:03 +0100
> schrieb lee :
>
>> Danny YUE  writes:
>> 
>> > On 2017-04-25 14:29, lee  wrote:  
>> >> Hi,
>> >>
>> >> since the usage of FTP seems to be declining, what is a replacement
>> >> which is at least as good as FTP?
>> >>
>> >> I'm aware that there's webdav, but that's very awkward to use and
>> >> missing features.  
>> >
>> > What about sshfs? It allows you to mount a location that can be
>> > accessed via ssh to your local file system, as if you are using
>> > ssh.  
>> 
>> Doesn't that require ssh access?  And how do you explain that to ppl
>> finding it too difficult to use Filezilla?  Is it available for
>> Windoze?
>
> Both, sshfs and scp, require a full shell (that may be restricted but
> that involves configuration overhead on the server side).

I wouldn't want them to have that.

> You can use sftp (FTP wrapped into SSH), which is built into SSH. It
> has native support in many Windows clients (most implementations use
> PuTTY in the background). It also has the advantage that you can
> easily restrict users on your system to SFTP-only with an easy
> server-side configuration.

>From what I've been reading, sftp is deprecated and has been replaced by
ftp with TLS.

>> > Also samba can be a replacement. I have a samba server on my OpenWRT
>> > router and use mount.cifs to mount it...  
>> 
>> Does that work well, reliably and securely over internet connections?
>
> It supports encryption as transport security, and it supports kerberos
> for secure authentication, the latter is not easy to setup in Linux,
> but it should work with Windows clients out-of-the-box.
>
> But samba is a pretty complex daemon and thus offers a big attack
> surface for hackers and bots. I'm not sure you want to expose this to
> the internet without some sort of firewall in place to restrict access
> to specific clients - and that probably wouldn't work for your scenario.

At least it's a possibility.  I don't even know if they have static IPs,
though.

> But you could offer access via OpenVPN and tunnel samba through that.

I haven't been able yet to figure out what implications creating a VPN
has.  I understand it's supposed to connect networks through a secured
tunnel, but what kind of access to the LAN does someone get who connects
via VPN?  Besides, VPN is extremely complicated and difficult to set
up.  I consider it an awful nightmare.

Wireguard seems a lot easier.

> By that time, you can as easily offer FTP, too, through the tunnel
> only, as there should be no more security concerns now: It's encrypted
> now.

The ftp server already doesn't allow unencrypted connections.

Now try to explain to ppl for whom Filezilla is too complicated how to
set up a VPN connection and how to secure their LAN once they create the
connection (if we could ever get that to work).  I haven't been able to
figure that out myself, and that is one of the main reasons why I do not
have a VPN connection but use ssh instead.  The only disadvantage is
that I can't do RDP sessions with that ---  I probably could and just
don't know how to --- but things might be a lot easier if wireguard
works.

> OpenVPN also offers transparent compression which can be a big
> plus for your scenario.

Not really, a lot of data is images, usually JPEG, some ZIP files, some
PDF.  All that doesn't compress too well.

> OpenVPN is not too difficult to setup, and the client is available for
> all major OSes. And it's not too complicated to use: Open VPN
> connection, then use your file transfer client as you're used to. Just
> one simple extra step.

I'm finding it a horrible nightmare, see above.  It is the most
difficult thing you could come up with.  I haven't found any good
documentation that explains it, the different types of it, how it works,
what to use (apparently there are many different ways or something, some
of which require a static IP on both ends, and they even give you
different disadvantages in performance ...), how to protect the
participants and all the complicated stuff involved.  So far, I've
managed to stay away from it, and I wouldn't know where to start.  Of
course, there is some documentation, but it is all confusing and no
good.

The routers even support it.  In theory, it shouldn't be difficult to
set up, but that's only theory.  They do not have any documentation as
to how to protect the connected networks from each other.  I could
probably get it to work, but I wouldn't know what I'm doing, and I don't
like that.


I admit that I don't really want to know how VPN works because it's
merely an annoyance and not what I need.  What's needed is a simple,
encrypted connection between networks, and VPN is anything but that.

Wireguard sounds really simple.  Since I need to set up a VPN or
VPN-like connection sooner than later, I'm considering using it.


-- 
"Didn't work" is an error.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Sat, 29 Apr 2017 20:02:57 +0100
> schrieb lee :
>
>> Alan McKinnon  writes:
>> 
>> > On 25/04/2017 16:29, lee wrote:  
>> >> 
>> >> Hi,
>> >> 
>> >> since the usage of FTP seems to be declining, what is a replacement
>> >> which is at least as good as FTP?
>> >> 
>> >> I'm aware that there's webdav, but that's very awkward to use and
>> >> missing features.
>> >> 
>> >>   
>> >
>> > Why not stick with ftp?  
>> 
>> The intended users are incompetent, hence it is too difficult to
>> use ...
>
> If you incompetent users are using Windows: Have you ever tried
> entering ftp://u...@yoursite.tld in the explorer directory input bar?

I tried at work and it said something like that the service cannot be
accessed.


> [...]
> Debian is not the king to rule the internet. You shouldn't care when
> they shut down their FTP services. It doesn't matter to the rest of the
> world using the internet.

Who can say what their influence actually is?  Imagine Debian going
away, and all the distributions depending on them as well because they
loose their packet sources, then what remains?  It is already rather
difficult to find a usable distribution, and what might the effect on
upstream sources be.

>> > There's always dropbox  
>> 
>> Well, dropbox sucks.  I got a dropbox link and it didn't work at all,
>> and handing out the data to some 3rd party is a very bad idea.  It's
>> also difficult to automate things with that.
>
> There's also owncloud (or whatever it is called now). You can automate
> things by deploying a sync application on your clients side.

The problem is that they would need to do that themselves.  It would be
much easier to show them how to use Filezilla ...


-- 
"Didn't work" is an error.

Re: [gentoo-user] Re: replacement for ftp?

[Poison BL.]

On Tue, May 2, 2017 at 12:01 PM, Ian Zimmerman  wrote:

> On 2017-05-02 09:05, Neil Bothwick wrote:
>
> >> miles per hour, internet almost always won lately. :-)
>
> > But tapes also hold a lot more than they did in 1981
>
> And so do trucks :-(
>
> --
> Please *no* private Cc: on mailing lists and newsgroups
> Personal signed mail: please _encrypt_ and sign
> Don't clear-text sign:
> http://primate.net/~itz/blog/the-problem-with-gpg-signatures.html
>
>
Not just trucks loaded with tapes... trucks loaded full of MicroSD cards
are getting downright silly, density-wise... and the bandwidth of that
running down a highway still wins.

-- 
Poison [BLX]
Joshua M. Murphy

[gentoo-user] Re: replacement for ftp?

[Ian Zimmerman]

On 2017-05-02 09:05, Neil Bothwick wrote:

>> miles per hour, internet almost always won lately. :-)

> But tapes also hold a lot more than they did in 1981

And so do trucks :-(

-- 
Please *no* private Cc: on mailing lists and newsgroups
Personal signed mail: please _encrypt_ and sign
Don't clear-text sign:
http://primate.net/~itz/blog/the-problem-with-gpg-signatures.html

Re: [gentoo-user] Re: replacement for ftp?

[Neil Bothwick]

On Sun, 30 Apr 2017 08:22:50 +0200, Kai Krakow wrote:

> > > Never underestimate the bandwidth of a station wagon full of tapes
> > > hurtling down the highway.
> 
> Hehe, with the improvements in internet connections nowadays, we
> almost stopped transferring backups via sneakernet. Calculating the
> transfer speed of the internet connection vs. the speed calculating
> miles per hour, internet almost always won lately. :-)

But tapes also hold a lot more than they did in 1981, so bandwidth is
increasing on both sides ;-)


-- 
Neil Bothwick

QOTD:
The only easy way to tell a hamster from a gerbil is that the
gerbil has more dark meat.


pgpZTu80PEPJK.pgp
Description: OpenPGP digital signature

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sat, 29 Apr 2017 22:02:51 -0400
schrieb "Walter Dnes" :

>   Then there's always "sneakernet".  To quote Andrew Tanenbaum from
> 1981
> 
> > Never underestimate the bandwidth of a station wagon full of tapes
> > hurtling down the highway.  

Hehe, with the improvements in internet connections nowadays, we
almost stopped transferring backups via sneakernet. Calculating the
transfer speed of the internet connection vs. the speed calculating
miles per hour, internet almost always won lately. :-)

Most internet connections are faster than even USB sticks these days.
LAN connections tailored towards storage are even fast then ordinary
hard disks (i.e. SAN).

Nice fun fact, tho. If go "a station wagon full", it probably still
holds true today.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sat, 29 Apr 2017 20:02:57 +0100
schrieb lee :

> Alan McKinnon  writes:
> 
> > On 25/04/2017 16:29, lee wrote:  
> >> 
> >> Hi,
> >> 
> >> since the usage of FTP seems to be declining, what is a replacement
> >> which is at least as good as FTP?
> >> 
> >> I'm aware that there's webdav, but that's very awkward to use and
> >> missing features.
> >> 
> >>   
> >
> > Why not stick with ftp?  
> 
> The intended users are incompetent, hence it is too difficult to
> use ...

If you incompetent users are using Windows: Have you ever tried
entering ftp://u...@yoursite.tld in the explorer directory input bar?

> > Or, put another way, why do you feel you need to use something
> > else?  
> 
> I don't want to use anything else.
> 
> Yet even Debian has announced that they will shut down their ftp
> services in November, one of the reasons being that almost no one uses
> them.  Of course, their application is different from what I'm looking
> for because they only have downloads and no uploads.

And that's the exact reason why: Offering FTP just for downloads (not
even for browsing) is inefficient. Getting a file via HTTP is much more
efficient as the connection overhead is much lower. Removing FTP is
thus just a question of reducing attack surface and server load.

Your scenario differs a lot and doesn't follow the reasoning debian put
behind it.

> However, another reason given was that ftp isn't exactly friendly to
> firewalls and requires "awkward kludges" when load balancing is used.
> That is a pretty good reason.

This is due to FTP incorporating transfer of ports and IP addresses in
the protocol which was a good design decision when the protocol was
specified but isn't nowadays. Embedding FTP into a tunnel solves that,
e.g. by using sftp (ssh+ftp). HTTP also solves that by not embedding
such information at the protocol level. But tunneling FTP is not how
you would deploy such a scenario, so the option is HTTP, hence FTP can
be shut down by debian. KISS principle.

> Anyway, when pretty much nobody uses a particular software anymore, it
> won't be very feasible to use that software.

Nobody said that when debian announced to shut down their FTP servers.
Debian is not the king to rule the internet. You shouldn't care when
they shut down their FTP services. It doesn't matter to the rest of the
world using the internet.

> > There's always dropbox  
> 
> Well, dropbox sucks.  I got a dropbox link and it didn't work at all,
> and handing out the data to some 3rd party is a very bad idea.  It's
> also difficult to automate things with that.

There's also owncloud (or whatever it is called now). You can automate
things by deploying a sync application on your clients side.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sat, 29 Apr 2017 20:30:03 +0100
schrieb lee :

> Danny YUE  writes:
> 
> > On 2017-04-25 14:29, lee  wrote:  
> >> Hi,
> >>
> >> since the usage of FTP seems to be declining, what is a replacement
> >> which is at least as good as FTP?
> >>
> >> I'm aware that there's webdav, but that's very awkward to use and
> >> missing features.  
> >
> > What about sshfs? It allows you to mount a location that can be
> > accessed via ssh to your local file system, as if you are using
> > ssh.  
> 
> Doesn't that require ssh access?  And how do you explain that to ppl
> finding it too difficult to use Filezilla?  Is it available for
> Windoze?

Both, sshfs and scp, require a full shell (that may be restricted but
that involves configuration overhead on the server side). You can use
sftp (FTP wrapped into SSH), which is built into SSH. It has native
support in many Windows clients (most implementations use PuTTY in the
background). It also has the advantage that you can easily restrict
users on your system to SFTP-only with an easy server-side
configuration.

> > Also samba can be a replacement. I have a samba server on my OpenWRT
> > router and use mount.cifs to mount it...  
> 
> Does that work well, reliably and securely over internet connections?

It supports encryption as transport security, and it supports kerberos
for secure authentication, the latter is not easy to setup in Linux,
but it should work with Windows clients out-of-the-box.

But samba is a pretty complex daemon and thus offers a big attack
surface for hackers and bots. I'm not sure you want to expose this to
the internet without some sort of firewall in place to restrict access
to specific clients - and that probably wouldn't work for your scenario.

But you could offer access via OpenVPN and tunnel samba through that.
By that time, you can as easily offer FTP, too, through the tunnel
only, as there should be no more security concerns now: It's encrypted
now. OpenVPN also offers transparent compression which can be a big
plus for your scenario.

OpenVPN is not too difficult to setup, and the client is available for
all major OSes. And it's not too complicated to use: Open VPN
connection, then use your file transfer client as you're used to. Just
one simple extra step.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Sat, 29 Apr 2017 20:38:24 +0100
schrieb lee :

> Kai Krakow  writes:
> 
> > Am Tue, 25 Apr 2017 15:29:18 +0100
> > schrieb lee :
> >  
> >> since the usage of FTP seems to be declining, what is a replacement
> >> which is at least as good as FTP?
> >> 
> >> I'm aware that there's webdav, but that's very awkward to use and
> >> missing features.  
> >
> > If you want to sync files between two sites, try rsync. It is
> > supported through ssh also. Plus, it's very fast also.  
> 
> Yes, I'm using it mostly for backups/copies.
> 
> The problem is that ftp is ideal for the purpose, yet users find it
> too difficult to use, and nobody uses it.  So there must be something
> else as good or better which is easier to use and which ppl do use.

Well, I don't see how FTP is declining, except that it is unencrypted.
You can still use FTP with TLS handshaking, most sites should support
it these days but almost none forces correct certificates because it is
usually implemented wrong on the server side (by giving you
ftp.yourdomain.tld as the hostname instead of ftp.hostingprovider.tld
which the TLS cert has been issued for). That makes it rather pointless
to use. In linux, lftp is one of the few FTP clients supporting TLS
out-of-the-box by default, plus it forces correct certificates.

But I found FTP being extra slow on small files, that's why I suggested
to use rsync instead. That means, where you could use sftp (ssh+ftp),
you can usually also use ssh+rsync which is faster.

There's also the mirror command in lftp, which can be pretty fast, too,
on incremental updates but still much slower than rsync.

> I don't see how they would transfer files without ftp when ftp is the
> ideal solution.

You simply don't. FTP is still there and used. If you see something
like "sftp" (ssh+ftp, not ftp+ssl which I would refer to as ftps), this
is usually only ftp wrapped into ssh for security reasons. It just
using ftp through a tunnel, but to the core it's the ftp protocol. In
the end, it's not much different to scp, as ftp is really just only a
special shell with some special commands to setup a file transfer
channel that's not prone to interact with terminal escape sequences in
whatever way those may be implemented, something that e.g. rzsz needs
to work around.

In the early BBS days, where you couldn't establish a second transfer
channel like FTP does it using TCP, you had to send special escape
sequences to put the terminal into file transfer mode, and then send
the file. By that time, you used rzsz from the remote shell to initiate
a file transfer. This is more the idea of how scp implements a file
transfer behind the scenes.

FTP also added some nice features like site-to-site transfers where the
data endpoints both are on remote sites, and your local site only is
the control channel. This directly transfers data from one remote site
to another without going through your local connection (which may be
slow due to the dial-up nature of most customer internet connections).

Also, FTP is able to stream multiple files in a single connection for
transferring many small files, by using tar as the transport protocol,
thus reducing the overhead of establishing a new connection per file.
Apparently, I know only few clients that support that, and even fewer
servers which that would with.

FTP can be pretty powerful, as you see. It's just victim of its poor
implementation in most FTP clients that makes you feel it's mostly
declined. If wrapped into a more secure tunnel (TLS, ssh), FTP is still
a very good choice for transferring files, tho not the most efficient.
Depending on your use case, you get away much better using more
efficient protocols like rsync.


-- 
Regards,
Kai

Replies to list-only preferred.

Re: [gentoo-user] Re: replacement for ftp?

[lee]

Kai Krakow  writes:

> Am Tue, 25 Apr 2017 15:29:18 +0100
> schrieb lee :
>
>> since the usage of FTP seems to be declining, what is a replacement
>> which is at least as good as FTP?
>> 
>> I'm aware that there's webdav, but that's very awkward to use and
>> missing features.
>
> If you want to sync files between two sites, try rsync. It is supported
> through ssh also. Plus, it's very fast also.

Yes, I'm using it mostly for backups/copies.

The problem is that ftp is ideal for the purpose, yet users find it too
difficult to use, and nobody uses it.  So there must be something else
as good or better which is easier to use and which ppl do use.  I don't
see how they would transfer files without ftp when ftp is the ideal
solution.


-- 
"Didn't work" is an error.

[gentoo-user] Re: replacement for ftp?

[Grant Edwards]

On 2017-04-27, Danny YUE  wrote:

> I recently found that rsnapshot (based on rsync) is a good and solid
> tool for backup...You may try it out ;-)

I second the recommendation for rsnapshot.  I've been using rsnampshot
for several years to backup stuff to an external Firewire RAID array.
The really convenient thing about it is that backup is simply a set of
directory trees that you can peruse at any time to verify that backups
are occuring or to look at old versions of files.

-- 
Grant Edwards   grant.b.edwardsYow! I'm having BEAUTIFUL
  at   THOUGHTS about the INSIPID
  gmail.comWIVES of smug and wealthy
   CORPORATE LAWYERS ...

[gentoo-user] Re: replacement for ftp?

[Kai Krakow]

Am Tue, 25 Apr 2017 15:29:18 +0100
schrieb lee :

> since the usage of FTP seems to be declining, what is a replacement
> which is at least as good as FTP?
> 
> I'm aware that there's webdav, but that's very awkward to use and
> missing features.

If you want to sync files between two sites, try rsync. It is supported
through ssh also. Plus, it's very fast also.

-- 
Regards,
Kai

Replies to list-only preferred.