Notes from mounting Android folder over wifi and making backups to Linux machine automatic [2015-10-25]

Author: Grzegorz Wierzowiecki

Writing style remarks

Hi! As I was not writing such tutorial to general audience, I had trouble in balancing tradeoff about assumptions what’s obvious, what’s not to potential reader. I hope you will enjoy, I am happy to receive suggestions, might be in form of git patches against this file (as it’s hosted on github).

Intro:

Surprisingly, it took more time to find right Android app to mount photos directory of Android device over wifi (not usb cable) as Linux directory.

As I spend a bit of time trying few solutions, this inspired me to share.

Here, I share what I found most robust and flexible to me.
Additionally I extend with a bit of comments that might make those notes helpful also for Linux+Android beginners.

Hopefully this note will let you enjoy your Android files after few minutes :).

Final goal:

My initial motivations:

Requirements and preferences:

Means / Implemented with:

Combination that worked for me:

How To

Linux - Cryptographic Key-Pair generation:

$ ssh-keygen -t rsa -b 8192 -f ~/.ssh/id_rsa_nopass
# * empty password: as I want to use it for automatic passwordless authentication I leave password empty
# * type RSA: as SSH Server on Android supports only RSA and DSA	
# Now, add key to ssh-agent so it can be used by ssh and sshfs
$ eval $(ssh-agent)
$ ssh-add ~/.ssh/id_rsa_nopass
# this generates both:
#  * private key: id_rsa_nopass
#  * public key: id_rsa_nopass.pub
# please keep your private key in secret only on your machine and do not transmit
# public key is the only key for public (other people) announcement

Android - Setting up SFTP server:

If section looks lengthy - You can just try do it or follow screenshots!

If you are not scared of lengthy description, let’s go over all options I set, step by step:

Linux - Finding IP address of device

Scanning local network with nmap

My approach is following. As I know that all my devices will run [SFTP] server on some defined and relatively unique to my environment port (let’s use 43210 as example), I decided to scan all internal IPs with nmap:

# Assuming your network is 192.168.0.0/24 (ifconfig is your friend)
nmap 192.168.0.0/24 -p 43210 --max-retries 1 --open

To automate this in my scripts I iterate in a loop over all devices with defined port open:

for ip in $(nmap 192.168.0.0/24 -p 43210 --max-retries 1 --open | grep -o '\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}'); do
  # magic explained below...
done

General network -> Use Dynamic DNS

I’ve chosen SSH Server partly due to it’s ability to register IP address via DDNS. However this option I did not yet explored it with this application or written results down, so I encourage, dear readers, to give it a try! (Or maybe even suggest patch to this article as it is github!)

Linux - mounting ssh server despite deprecated ssh-dss key type…

And does not mount:

read: Connection reset by peer

Just mounting with -d debug mode results in :

nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
Unable to negotiate with 192.168.0.123: no matching host key type found. Their offer: ssh-dss
read: Connection reset by peer

And now, we know what has happened. As we know from OpenSSH 7.0 announcement ssh-dss keys are deprecated since then and we need to follow OpenSSH legacy instructions to use it, to sum it up: to bypass it we need to pass -oHostKeyAlgorithms=+ssh-dss flag to ssh. (Yes, I know it sucks, but I really didn’t find better SFTP server for android offering more up-to-date crypto and restriction to just read-only ftp (i.e. “no shell”), if you find sth better, please let me know. For now, I consider this still ok for my use case, so let’s continue).

Therefore, let’s run sshfs with following assumptions:

$ sshfs AwesomeUsername@192.168.0.123:/storage/emulated/legacy/DCIM /path_we_mount -p 43210 -o uid=$(id -u),gid=$(id -g) -o HostKeyAlgorithms=+ssh-dss

Double check contents of directory!

$ ls -lha /path_we_mount /Camera | tail -n 4
-r-------- 1 gwpl users 153M Oct 20 20:22 VID_20151020_202055.mp4
-r-------- 1 gwpl users 346M Oct 22 23:27 VID_20151022_232428.mp4
-r-------- 1 gwpl users 205M Oct 22 23:29 VID_20151022_232735.mp4
-r-------- 1 gwpl users  52M Oct 23 11:57 VID_20151023_115641.mp4

Making backup with rsync

Now you can run rsync in dry-run (it means, it will not make any changes, just write what would do):

$ rsync --dry-run --recursive --times --progress \
  --exclude '*.thumbnails*' \
  --exclude '*Camera/.aux/.nomedia' \
  --exclude '*Camera/.aux/*' \
  --exclude '*Camera/*.mp4.tmp' \
  --exclude '*Camera/thumbnails/*' \
   /path_we_mount \
   /path_where_to_copy

I encourage to learn about more flags in man rsync (my usual set if -aAXv), here short survival summary for photo backups:

If output of dry run looks fine and operations such we want to happen, then remove --dry-run flag and run command again, to actually perform operations:

$ rsync --bwlimit=4096KiB  --dry-run --recursive --times --progress \
  --exclude '*.thumbnails*' \
  --exclude '*Camera/.aux/.nomedia' \
  --exclude '*Camera/.aux/*' \
  --exclude '*Camera/*.mp4.tmp' \
  --exclude '*Camera/thumbnails/*' \
  /path_we_mount \
  /path_where_to_copy

(As you see I limited bandwidth, so give phone and wifi more breathing, but you are free to drop limits.)

Cleanup -> let’s umount device

After everything, let’s umount (opposite to earlier mount):

$ fusermount -u /path_we_mount