• ARP-A-HOST, a script to resolve local network hostnames without extra services

    I was in desperate need to find a way to not always have to scan the network or go over to another device to know its IP so I could ssh into it or use a service it was serving. This is what I found.

    The internet is just a lot of devices, which can be addressed with an IP address. Those are unique in every network. To make the network more usable we use domain names with logical, easy to remember names like www.google.com or www.facebook.com.

    This works by contactin a server with a fixed IP which provides DNS that resolves your domain name in an IP that your browser can go to. DNS-servers for the global internet are managed by governments and maintained by big companies like Google. This is why, If you want a domain name, it comes with a fee…

    On local networks, its overkill (and quite some setup) to have a DNS-server to resolve the IP of devices but other programs like zeroconf and mdns tried to have a more low-maintenance take on this problem. Short story: Still not ideal. You still have to have services running on all devices to ensure that your device gets registered…

    The solution

    The ideal solution consists of:

    – No changes needed to the network (so it works everywhere)

    – Reliable

    – Very little setup on the device itself and no services need to be running

    – Cross-platform

    The idea is to maintain a list of MAC-addresses of the devices I use (and sync those with Dropbox or Git), scan the network for these MAC-addresses and if found, add them to the /etc/hosts file. This is reliable because MAC-addresses are hard-coded in your hardware. You only need to know the MAC of your other devices and you can identify it. Every device has to have one. Adding to the /etc/hosts file is something that is supported by the core of linux so it is quite cross-platform on any device you can get linux working on.

    So whenever my network-setup changes, or it has been a while and adresses may have switched, I run the following command;

    bash ./ARP-A-HOST.sh ./HOSTS_MACS

    ARP-A-HOST.sh script:

    #DEPENDENCIES: arp-scan
    #take $1 (first argument of the script) -> file with mac adresses + hostname
    
    tmpfile=$(mktemp /tmp/DYNAMIC-RESOLVE.XXXXXX) #tmp file for temporary results
    
    #clear lines form the previous run in /etc/hosts file
    sed -i.bak '/DYNAMIC_RESOLVE/d' /etc/hosts
    
    #use the tool arp-scan to find all the devices on the network
    if ! [ -z $2 ]; then
     arp-scan -l --localnet --interface=$2 >> $tmpfile
    else
     for line in $(ip link | cut -d " " -f 2); do
     interface=${line::-1}
     arp-scan -l --localnet --interface=$interface 2>/dev/null >> $tmpfile
     done
     #TODO interfaceoption -> default wlan
    fi
    echo Arpscan Finished... Filtering results...
    
    #reading the results one by one
    while read -r line
    do
     MAC=$(echo $line | cut -d " " -f 1)
     NAME=$(echo $line | cut -d " " -f 2)
     #grep MAC from ARP cache
     IP=$(cat $tmpfile | grep $MAC | cut -d$'\t' -f 1)
    
    if ! [ -z "${IP}" ]; then
     echo Found $NAME at $IP! Adding to /etc/hosts...
     echo "$IP $NAME $NAME #DYNAMIC_RESOLVE">>/etc/hosts #If the MAC is found, add to the /etc/hosts file
     fi 
    done < "$1"
    echo Done with discovering hosts

    HOSTS_MACS file:

    00:90:f5:d6:5b:05 c1
    c0:ee:fb:59:fc:23 s1
    10:02:b5:d6:08:8a o1
    b8:27:eb:4e:0c:42 kh1

    The second file is just a list of MAC-addresses with a name you chose behind it. You can find your MAC-address in various ways, but the easiest is just running the ifconfig command on linux. (look next to “ether”)

    [root@localhost]# ifconfig
    wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 192.168.1.13  netmask 255.255.255.0  broadcast 192.168.1.255
            ether 10:02:b5:d6:08:8a  txqueuelen 1000  (Ethernet)
            RX packets 221361  bytes 303819415 (289.7 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 112644  bytes 15169217 (14.4 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    ...
    

    You will need the tool arp-scan to run this script, you can find it in nearly all repositories. I run this script on an android phone, a desktop, chromebook and Raspberry Pi. So no worries.

    If you check the /etc/hosts file you can see the following has changed:

    #
    # /etc/hosts: static lookup table for host names
    #
    
    #<ip-address> <hostname.domain.org> <hostname>
    127.0.0.1 localhost.localdomain localhost
    ::1 localhost.localdomain localhost
    
    # End of file
    192.168.1.3 s1 s1 #DYNAMIC_RESOLVE
    192.168.1.2 kh1 kh1 #DYNAMIC_RESOLVE

    You can now use the hostnames in most commands and have them automatically resolved;

    ssh s1 #resolves to the IP adress of my phone
    ssh root@kh1 -p 22 #same but with more arguments

    You can alias the ARP-A-HOST.sh-script command in you bashrc](https://demgeeks.com/qt-make-the-command-line-easier-with-aliases-and-functions/) to make it easier to use. There is also a Github-repo that contains all the files needed.

    That’s it! have fun!

  • Setting Up a GnuPG-based Password Manager

    Header

    In this article we will setup a secure password manager. You probably use the same password over and over again on multiple sites/devices/applications. Not so great! If one service gets compromised, an attacker can wreak some serious havoc. YOU SHOULD USE DIFFERENT PASSWORDS!

    But this is hard, so many store their passwords in a passwords.txt file on their desktop, or even better, in a service like Dropbox storing all passwords together and keeping that file synced. Copying passwords will also circumvent keyloggers active on your system. So awesome right?!

    Not really, If one of your devices gets compromised, or Dropbox makes a mistake, you are screwed. You should still have an extra layer of access protection between the file and the passwords, syncing with Dropbox can be fine, but you have to encrypt the file.

    About password managers

    One option is using tools like Keepass or 1Password. But you are depending on that service, its not modular (you can not choose the way it gets synced). I don’t like trusting these services. Not only the communication but things like encryption type, memory leaks, temporary files can all be badly designed.  Open-source tools are the way to go, because you can trust AND verify.

    The best way to do this is with asymmetric encryption. Their are standards such as PGP (GnuPG on linux) who make this easy. The encryption is very strong because you use a long encryption key, to decrypt your target file.

    The long encryption key is stored locally on your machines, and you need a passphrase to use it. So even if the can get access to the long secure encryption key (private key). They will still need the passphrase.

    This is not all that GnuPG can do, it is an amazing piece of work, if interested, you should check it out. A nice place to get started with using it is this Arch Linux page about GnuPG.

    I use the terminal everywhere and there is a cli-tool named pass to automate the usage of GnuPG for passwords. We will use that to manage our passwords. The cool thing about the pass password-manager, is its cross platform uniform behavior. You know how to use it on one platform and you know it on all your other devices (more on syncing the passwords later)

    Dependencies

    You first have to install pass and GnuPG and more important, set up GnuPG and have it make a key for you (you can of course use an already existing key, but then you would not be reading this). Do what is right on your system:

    Debian-based;

    sudo apt-get install pass gnupg
    

    Arch Linux; 

    sudo pacman -Sy pass gnupg
    

    Setup your encryption key

    gpg --full-gen-key
    

    Now gpg will ask you a couple questions, like what kind of key you want, press 1. I chose a key-size of 4096, which is the highest possible. Then put in 0 for a everlasting key, confirm you choice. Then it will ask for a name, email and comment (like “created pm 27/12/2016”). Confirm with pressing O and ENTER. Then STOP!

    You should now have the following output:

    [root@localhost ~]# gpg --full-gen-key
    gpg (GnuPG) 2.2.4; Copyright (C) 2017 Free Software Foundation, Inc.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    
    Please select what kind of key you want:
       (1) RSA and RSA (default)
       (2) DSA and Elgamal
       (3) DSA (sign only)
       (4) RSA (sign only)
    Your selection? 1
    RSA keys may be between 1024 and 4096 bits long.
    What keysize do you want? (2048) 4096
    Requested keysize is 4096 bits
    Please specify how long the key should be valid.
             0 = key does not expire
          <n>  = key expires in n days
          <n>w = key expires in n weeks
          <n>m = key expires in n months
          <n>y = key expires in n years
    Key is valid for? (0) 0
    Key does not expire at all
    Is this correct? (y/N) y
    
    GnuPG needs to construct a user ID to identify your key.
    
    Real name: lvlrt
    Email address: [email protected]
    Comment: Thisismycomment
    You selected this USER-ID:
        "lvlrt (Thisismycomment) <[email protected]>"
    
    Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
    We need to generate a lot of random bytes. It is a good idea to perform
    some other action (type on the keyboard, move the mouse, utilize the
    disks) during the prime generation; this gives the random number
    generator a better chance to gain enough entropy.
    Please enter the passphrase to
    protect your new key
    Passphrase: 
    

    Now put a YouTube video playing, setup a file copy from and to your disk for 10 minutes or something. Then go back to your console and it will have asked for a password. Fill in the password while your device is doing the heavy work. Confirm and you are done.

    You can close your YouTube video and the file transfer. This was to generate some computer noise. Which will improve the randomness of the generated key!

    To test if the key was successfully created enter the following command:

    [root@localhost ~]# gpg --list-keys
    ...
    ----------------------------
    pub   rsa2048 2018-01-10 [SC]
          927A1402D78C9188216F233841CB4088BBF533E9
    uid           [ultimate] lvlrt (test) <[email protected]>
    

    Ok great!, let us now get pass setup, you can do that with this command: (where the <id> value is your email or name you just put in)

    pass init <id>
    

    Then it will output the following:

    [root@localhost ~]# pass init [email protected]
    mkdir: created directory '/home/king/.password-store/'
    Password store initialized for [email protected]
    

    Setup done!

    Usage

    To insert a password in your password-store:

    pass insert stuffathome/thingseverybodyshouldknow/wifipassword

    You can keep adding or removing ‘/’ signs to make more divisions in your password-store.

    These will just be added as descending folders.

    To have it instead generate a password for you (if you want to be extra safe) and store it in an encrypted file:

    pass generate stuffathome/thingseverybodyshouldknow/wifipassword

    No password asked? Isn’t this unsafe?

    The way we encrypted this password-store is asymmetrical… so everybody can encrypt, but only the ones with a private key can decrypt.

    To list the available passwords just type:

    pass show

    To retrieve the password we just created, do:

    pass show stuffathome/thingseverybodyshouldknow/wifipassword

    And now it will ask for your passphrase of the gpg key you specified in the setup command. Then it should echo the password out. And thats really all you need to know to use this password manager. The key will be available for a small time until it timed out. That’s it!

    Syncing your passwords

    To synchronize your password manager between devices. You need to synchronize 2 sets of files. The first one is you gpg key. Those are stored in the folder ~/.gnupg by default. You can change the location where GnuPG checks for these files with setting the $GNUPGHOME variable:

    export GNUPGHOME="~/keys"

    Now you can move your .gnupg folder to a new location and put the new location in this variable. REMINDER: you have to put this command in your ~/.bashrc file or alias it with your pass command (more on that later).

    You passwords are stored in a directory, called a store, this is the one that pass has setup for you. The default location is ~/.password-store. If you use change the variables $PASSWORD_STORE_DIR and $PASSWORDS_STORE_GIT to the location where your store is. So all this in one command (put this in ~/.bashrc):

    alias pass='GNUPGHOME=~/keys PASSWORD_STORE_DIR=~/DATA/PASSWORDS PASSWORD_STORE_GIT=~/DATA/PASSWORDS pass'

    This alias command, will make your pass command always use these variables. You copy the passwords-store and key to a service like Dropbox or in a synced git-repository, put the path to these directories in the command and you are set! Please keep the key and password-store seperate, it is still protected by a passphrase but you lose the protection of you 4096-bit key.

    That’s it! Good Luck with your password manager!

  • A transparent poor man's VPN with sshuttle

    header

    Why?

    There are a lot of reasons why you would want a proxy or VPN, one of them is safety and protection from attackers at your location. For example if you are visiting sensitive things in your local coffee shop, other visitors could sniff your traffic because they are connected to the same network. From that point it is not only possible to read but also manupulate the ftraffic and possibly inject malicious code. Other usage cases are services who are only accessible behind a firewall or only if the user appears to be located at a certain specified IP address.

    Most of the time, a proxy is one connection that gets rerouted while a VPN is a more client-friendly option to reroute all traffic from a system. If you do not want to configure every application to use your proxy, and want to reroute all traffic, the tool shuttle is a perfect fit. It uses SSH under the hood so the permissions are managed through your existing SSH server so no extra user management is required.

    I you are searching for a way to only route one command or application through a proxy have a look at an earlier article I wrote. In a lot of cases you don’t need to search the applications native capability for configuring a proxy. The more traffic you route through your ssh server, the more demanding and slow the operation becomes. So keep that in mind. Let’s begin!

    How to install?

    There are 2 things you will need;

    • A ssh server on a remote device (there are tutorials specific to your flavour of linux)
    • pip, the package manager from python

    To install the python package manager, run one of the following commands;

    apt-get install python-pip # for debian and others
    pacman -Sy python pip #for archlinux users
    

    Now we can install sshuttle;

    pip install sshuttle
    

    Done!

    How to setup?

    There are a lot of general options and specific routing options that can be configured with sshuttle. But this article will just explain the transparent proxy (reroute all outgoing traffic). You can find the full documentation here.

    Now,for example, if we had an SSH server located at 192.168.13.2 and properly configured to accept connections from our address, we could run;

    sshuttle --dns -vvr [email protected]:22 0/0
    

    This will first make a connection to your ssh server and then start routing all traffic from your system to this location. As you can see from the extra option. Also your DNS-requests will be tunneled.

    To make sure there is not traffic leaked, look into network monitoring software like wireshark. To test launch some applications and see if there is info being leaked.

    Tip for experts

    You can make a bash function to automate the process and be able to quickly switch between servers. For this example we will use the basic sshuttle command, but any one of them will work.

    You can add a function to your .bashrc-file like this;

    proxy () {
    sshuttle --dns -vvr $1 0/0
    }
    

    The result of this will be that you can run one of the following in your bash-prompt:

    $ proxy [email protected]:22 
    $ proxy ssh_server1
    

    And it will proxy all your traffic to a preconfigured ssh server. To use your proxy as in the second example, you will have to configure your destination in the ~/.ssh/config file from your system to make it really useful It will even take the keys that are loaded in your SSH-agent.

    There is awesome documentation of this program, so make sure to check it out!

  • Quick tip: Running a terminal command through a proxy

    Sometimes, running a full VPN is not necessary, or there is not enough bandwidth for all your traffic. In cases of using Tor for example, tunneling all your traffic can even be dangerous! So there is a way to specify per command or program if you want to have it tunnel its web traffic through the proxy or not. This tool is called proxychains.

    If you, for example want to download a file through a proxy with the wget command. Just prepend the command with proxychains and done!

    $ proxychains wget www.remoteserver.com/fileIneed
    [proxychains] config file found: /etc/proxychains.conf
    [proxychains] preloading /usr/lib/libproxychains4.so
    [proxychains] DLL init: proxychains-ng 4.12
    --2018-01-08 23:18:33--  http://www.remoteserver.com/fileineed
    

    Great! that works, but gives some output, you can silence the extra output with the -q flag.

    proxychains -q ...
    

    To set it up on your system follow the following steps:

    1. Installation

    For example on Arch Linux do:

    pacman -Sy proxychains
    

    On Ubuntu or other Debian-based distro’s:

    apt-get install proxychains
    

    2. Configuration

    Proxychains has a lot of configuration options but all you need to do, is go to the end of the file /etc/proxychains.conf and edit the last line;

    #nano /etc/proxychains.conf
    socks4  127.0.0.1 9050
    

    It’s preconfigured to use tor, That means a socks4 proxy on localhost port 9050.

    Configure this to your needs, for example to use a SOCKS5 proxy made by SSH do this;

    • Command to run to make the SSH connection
      ssh remoteserver -D 5000
      
    • Edit the configuration file like this:
      socks5 127.0.0.1 5000
      

      3. Done!

    That’s it, proxies can be amazing to change your appearance to the public internet, get to otherwise inaccessible content or tunnel your way out of a restrictive firewall/filter. So knowing how to use them in a terminal enviroment is essential.