#22 xc (copy) client aborts copying 700MB file at approx. 500MB

Open
opened 2 years ago by Russtopia · 2 comments

TODO: gather details to reproduce, seen 2020-01-19 copying a large .mp4 file.

Client errors out with gzip: premature EOF encountered.

TODO: gather details to reproduce, seen 2020-01-19 copying a large .mp4 file. Client errors out with gzip: premature EOF encountered.
Russtopia commented 2 years ago
Owner

A very hacky workaround is implemented, for now, in branch https://gogs.blitter.com/RLabs/xs/src/xc-bigfile-EOF

This hack adds a 900ms delay after a copy has completed prior to finishing up the operation and setting CSExitStatus, closing the xsnet.Conn; giving remote end more time to close its end.

Suspected cause is that the sender dumps its data at full-speed, then immediately closes the channel after the tarpipe runs out of data. A better handshake between the sender and receiver, so that sender stays open until receiver has gotten all data and has closed its end, should be designed.

Perhaps server at end of copy should await a Finish msg from client before closing channel (with reasonable timeout of course, say, 10s)

A very hacky workaround is implemented, for now, in branch https://gogs.blitter.com/RLabs/xs/src/xc-bigfile-EOF This hack adds a 900ms delay after a copy has completed prior to finishing up the operation and setting CSExitStatus, closing the xsnet.Conn; giving remote end more time to close its end. Suspected cause is that the sender dumps its data at full-speed, then immediately closes the channel after the tarpipe runs out of data. A better handshake between the sender and receiver, so that sender stays open until receiver has gotten all data and has closed its end, should be designed. Perhaps server at end of copy should await a Finish msg from client before closing channel (with reasonable timeout of course, say, 10s)
Russtopia commented 2 years ago
Owner

As of 2020-02-26, branch xc-file-EOF is merged to master to mitigate this issue. The current mitigation is to add 900ms delay on the client end before exiting to give the remote side time for final data to go through.

The bug is seen more often with a remote host vs. tests with localhost, ie.

xc /tmp/someFile user@localhost:.   #'never' gives error 1026 vs.
xc /tmp/someFile user@example.org:. #occasionally gives error 1026

Transfers with error 1026 still seem to complete (no truncated data), but further testing is required.

Update 2021-01-31: error 1026 indeed seems to be non-fatal, no truncated files in months of use; but still a concern and merits testing with extremely high latency/turnaround connections. How to simulate?

As of 2020-02-26, branch xc-file-EOF is merged to master to mitigate this issue. The current mitigation is to add 900ms delay on the client end before exiting to give the remote side time for final data to go through. The bug is seen more often with a remote host vs. tests with localhost, ie. ``` xc /tmp/someFile user@localhost:. #'never' gives error 1026 vs. xc /tmp/someFile user@example.org:. #occasionally gives error 1026 ``` Transfers with error 1026 still seem to complete (no truncated data), but further testing is required. Update 2021-01-31: error 1026 indeed seems to be non-fatal, no truncated files in months of use; but still a concern and merits testing with extremely high latency/turnaround connections. How to simulate?
Sign in to join this conversation.
Loading...
Cancel
Save
There is no content yet.