This year, when going to DefCon once again, I figured I need a better VPN setup than I had been using previously. From digging around a bit, I found that Digitalocean has written a very easy-to-understand tutorial about how to set up OpenVPN and wanted to share it here as well. Here you go: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-openvpn-server-on-ubuntu-16-04

In order to get Android traffic going only over VPN, I recommend to use OpenVPN Connect app. Copy the key, cert and .opvn configuration on the device, and choose “Import” from the menu to easily set it up. I also recommend “Seamless tunnel” option in preferences that blocks any traffic until the VPN connection is established. Also make sure to check “Reconnect on reboot”.

Remember: if you go to DefCon, it’s much better to leave all of your electronic devices at home, including your smartphone (or any phone for that matter if you can). This setup is useful though if you a)know what you are doing b)want to go Chuck Norris style. That being said, even if you choose to use any devices, make sure that it contains, well, nothing, and that none of the content exchanged even via VPN is secretive as even the slightest mistake you make might make the whole setup futile.

Google App Engine lets you run cron jobs with the limitation that all of the jobs have to be accessed via API endpoints. Sometimes these jobs include sending requests to third party endpoints and can run for a long time. By default, App Engine kills the long request after 60 seconds, which is perfectly fine for user interaction, but for expected long-running job there has to be a better way (citing grand master Raymond Hettinger).

If you don’t expect it to be a regular HTTP response/request with client interaction, this limitation can be inconvenient. However, there is a way to work around it with TaskQueue. But even then, setting it up is possibly overcomplicating the solution for a simple job. Luckily, there is also deferred module.

For example, to allow a certain function to take longer, you can use the following code in bold:


from google.appengine.ext import deferred
from flask import Flask
from other_module import wrapped_long_method
app = Flask(__name__)

@app.route('/')
def hello():
    """Run a long job."""
    message = deferred.defer(wrapped_long_method)
    return message

Note that even though we are calling wrapped_long_method, it does not have () in the end of it. If you do need to pass parameters to the function, you can pass these as arguments to defer function, e.g.

deferred.defer(wrapped_long_method, parameter)

In order for this to work, app.yaml needs to have deferred turned on:

builtins:
- deferred: on

and add to the handlers section:

- url: /_ah/queue/deferred
    script: google.appengine.ext.deferred.deferred.application
    login: admin

After this your function and API endpoint call is not limited to 60 seconds.

To add a job to cron, create cron.yaml next to app.yaml with contect that suites your needs:

cron:
- description: cronjob that has a long running task
  url: /cron
  schedule: every 24 hours

This tutorial is originally meant to be a guide to go along with a talk for PyLadies (but also for anyone else who finds it useful) on how to create a simple Twilio SMS application that sends you a notification for an event. To make it more interesting, we’ll end up with an application that reminds you to bring your umbrella in the morning if it’s going to rain or apply sunscreen if the UV index for the day is going to be high.

Here is what you will need:

  • An editor
  • Python 2.7 or 3.5
  • Python twilio module: pip install twilio or easy_install twilio
  • A Twilio account, sign up here https://www.twilio.com/try-twilio. Don’t worry, go ahead, I’ll wait until you’re back. Make sure you verify your existing phone number, this is very important! Otherwise you won’t be able to send anything. It doesn’t make too much of a difference what you select for your possible projects to be, so choose whatever matches your interests the best.
  • A Twilio phone number that has capabilities for sending a SMS. Once you have registered, navigate to https://www.twilio.com/console/sms/dashboard, and click on the “Get started” button:
  • SMS dashboard

    To get a number:
    twilio2
    and it will offer you a number. If you don’t like this one, you can click on “Search for a different number”. In any way, make sure it has SMS listed in the capabilities (should be the case pretty much anywhere in the world). You should be greeted with a message similar to this:
    twilio3

    Congratulations, you how have a programmable number that has infinite use cases from notifying you when your train is late, telling you when there was movement detected by a camera, or sending your grandmother signs of affection that get sent out automatically at specific times so that she doesn’t feel lonely! Or make your dog send selfies to send them over MMS, your choice:
    lazlo
    Check out also this adorable video.

    You will also need the ACCOUNT_SID and AUTH_TOKEN to send a SMS. Keep in mind that the AUTH_TOKEN is equivalent to a password, so keep it secure accordingly. In order to get it, click on the “Show API credentials link”:
    twilio4
    or simply navigate to https://www.twilio.com/console/sms/getting-started/basics and click on the eye to see your token:
    twilio5
    The account sid always starts with AC and it’s an equivalent to a user name. Write both of these down, you will need it later.

    Ok, now we get to the fun part, coding. Sending a SMS is as simple as filling in the ACCOUNT_SID and AUTH_TOKEN variables with correct values and adjusting the message. Leave the quotes, replace just the text between these. Save it to a file, let’s say sms.py.

    from twilio.rest import TwilioRestClient
    
    # put your own credentials here
    ACCOUNT_SID = 'replace with your account sid'
    AUTH_TOKEN = 'replace with your account token'
    client = TwilioRestClient(ACCOUNT_SID, AUTH_TOKEN)
    
    client.messages.create(
        to = 'replace with the number you want to send a sms to',
        from_ = 'replace with your Twilio number you just created',
        body = 'Puppies! Frolicking kitties! Much fun',
    )

    Let’s try it out! Execute python sms.py from a shell and make sure that you either are in the directory where the file is or give it a full path. If it was successful, it has no output, you just get a new prompt line and you should receive a message with the contents you entered, padded with a free trial account message, like this:
    sms

    Congratulations, you have created your first Twilio SMS app! Now you can reuse this for any other application to send a notification for any sort of an event. Your imagination what to integrate it with is your playground. You could also execute the Python script from crontab periodically to check for any condition (let’s say a Twitter post for a specific user or a hashtag, stock price change and so on), and if it’s met, it will let you know by SMS.

    Note that you are not limited to SMS, you could also send MMS, make a call or use any of the other products available. https://www.twilio.com/docs/quickstart/python/sms and https://www.twilio.com/docs/tutorials are good places to explore next with great tutorials.

    In order to make the app a bit more interesting, let’s add UV index and weather (rain) monitoring so that it gives you a friendly nudge if you need to take action. Apply sunscreen when UV is higher than 6 and remind to take an umbrella with you in the morning if it’s going to rain. Pretty handy, huh?

    Here is the code:

    from twilio.rest import TwilioRestClient
    import requests
    import json
    
    # put your own credentials here
    ACCOUNT_SID = 'accountsid'
    AUTH_TOKEN = 'auth_token'
    client = TwilioRestClient(ACCOUNT_SID, AUTH_TOKEN)
    
    # ========================================================================
    # UV forecast 
    uv_url="https://iaspub.epa.gov/enviro/efservice/getEnvirofactsUVHOURLY/ZIP/94109/json"
    # fetch the data and load it to a json object
    uv_response = requests.get(uv_url)
    uv_data = json.loads(uv_response.text)
    # set initial values
    maximum_uv = -1
    uv_time = 0
    
    # loop over data
    for entry in uv_data:
        # if there is a higher value than current, remember its properties
        if entry['UV_VALUE'] > maximum_uv:
            maximum_uv = entry['UV_VALUE']
            uv_time = entry['DATE_TIME'].split()[1] \
                    + " " + entry['DATE_TIME'].split()[2]
    
    # ========================================================================
    # Rain forecast
    # this uses as API that gathers weather information from different sources
    url = "https://www.metaweather.com/api/location/2487956/"
    # fetch the data and load it to a json object
    response = requests.get(url)
    data = json.loads(response.text)
    # default value
    umbrella = False
    maxtemp = 0
    mintemp = 0
    rain_type = ''
    
    # loop over data
    for source in data['consolidated_weather']:
        # if any of the forecasts mentioned rain, bring an umbrella
        if "Rain" in source['weather_state_name'] or "Showers" in source['weather_state_name']:
            umbrella = True
            maxtemp = str(source['max_temp']).split(".")[0]
            mintemp = str(source['min_temp']).split(".")[0]
            rain_type = source['weather_state_name']
    
    # ========================================================================
    # send notifications if conditions are met
    if umbrella:
        rain = "You should bring an umbrella today, there is going to be " + rain_type + " and the temperature is between " + str(mintemp) + " and " + str(maxtemp) + "."
    
        client.messages.create(
           to = '+1tonumber',
           from_ = '+1twilionumber',
           body = rain,
        )
    
    if maximum_uv >= 6:
        message = "The maximum UV value today is " + str(maximum_uv) \
                + " at " + str(uv_time) \
                + ". Use sunscreen and try to stay indoors from 10 AM to 2 PM."
    
        client.messages.create(
           to = '+1tonumber',
           from_ = '+1twilionumber',
           body = message,
        )

    If you run it, it will send you a SMS immediately if either the UV index in a given day is too high in San Francisco (modify the location as needed if you live elsewhere) or it is a rainy day. While it is somewhat useful, it would be much better if it did the checking every morning on its own. For that in either Linux you can use crontab that will execute the Python script at your specified time. To configure it, run crontab -e, and add to the end of the file:
    0 8 * * * /usr/bin/python /path/to/sms.py
    and remember to look up where your python is located with which python and make sure the path to your application code is reachable via full path. A foolproof way to test it is to run ls /path/to/code to check if the file could be found at the location where you think it is. Also, this configuration would run the script every morning at 8 am, modify it according to your preferences. The first section is minutes, the second hours, then day of month, month and day of week. Using * (asterisk) in any of these places would execute it every time. For example, * for minutes would run in every minute, 0 * * * * would execute every hour, once per hour and so on. You can also specify intervals, */5 would be every 5 minutes.

    If you have any questions, let me know and I’ll try to help you out!

    Let’s say you want to automatically and nightly update all packages from the security repository. Beware that if you have updated any packages other ways than apt, it might not work for you. Run with superuser privileges:

    echo "0 4 * * * root test -x /usr/sbin/apt-security-updates && /usr/sbin/apt-security-updates" | tee --append /etc/cron.d/apt-security-updates

    Copy all security related lines to a separate apt file
    cat /etc/apt/sources.list |grep security > /etc/apt/sources.list.d/security.list

    And remove from the original file
    sed -i '/security/d' /etc/apt/sources.list

    Run apt-get update manually to check it works fine and there are no errors.

    Create the script that runs the update and set the permissions:
    echo 'date
    apt-get update
    apt-get upgrade -y -o Dir::Etc::sourcelist="sources.list.d/security.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0" -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"' | tee --append /usr/sbin/apt-security-updates
    chmod +x /usr/sbin/apt-security-updates

    The logs will be at /var/log/apt/history.log as for any other apt upgrade.

    In order to extract just one file from a deb package, you can do it like this:
    dpkg --fsys-tarfile package.deb| tar xOf - ./file/location/inside/deb > /path/to/destination
    For example:
    dpkg --fsys-tarfile libpng12-0_1.2.44-1+squeeze6_i386.deb| tar xOf - ./lib/libpng12.so.0.44.0 > ~/skype/usr/lib/libpng12.so.0.44.0
    Note that you do need to know the exact path and filename inside the package though. To list the contents:
    dpkg --contents package.deb

    While I needed to install Skype on Linux and even the multiarch package came up with
    dpkg: error processing archive skype-debian_4.3.0.37-1_i386 (1).deb (--install):
    package architecture (i386) does not match system (amd64)
    Errors were encountered while processing:
    skype-debian_4.3.0.37-1_i386.deb
    ,
    I thought I’d share my solution with others. This is written as of October 2016 with Skype 4.3.0. I am running Parrot Security OS, but since it’s a Debian derivative just as Ubuntu is for example, it’s likely going to work on other Debian derivatives just as well.

    I found this instructions page: https://support.skype.com/en/faq/FA12120/getting-started-with-skype-for-linux
    But.. these instructions are out of date as Squeeze repositories have been discontinued. However, this is how I got it working:
    So here goes:

  • Download a multiarch version from here: https://www.skype.com/en/download-skype/skype-for-linux/or-linux/” target=”_blank”>https://www.skype.com/en/download-skype/skype-for-linux/
  • mkdir -p ~/skype/usr/lib; dpkg -x ~/Downloads/skype-debian_4.3.0.37-1_i386.deb ~/skype/
    sudo dpkg --add-architecture i386; sudo apt-get update
    Install the dependencies:
    sudo apt-get install lib32asound2 ia32-libs ia32-libs-gtk libqt4-network:i386 libxi6:i386 libsqlite3-0:i386 libqt4-xmlpatterns:i386 libxv1:i386 libxss1:i386 libqtwebkit4:i386

    And check that everything is needed to make your Skype happy (there shouldn’t be any => not found):
    ldd ~/skype/usr/bin/skype
    linux-gate.so.1 (0xf77a6000)
    libXv.so.1 => /usr/lib/i386-linux-gnu/libXv.so.1 (0xf54ee000)
    libXss.so.1 => /usr/lib/i386-linux-gnu/libXss.so.1 (0xf54eb000)
    librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0xf54e2000)
    libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf54dd000)
    libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xf538b000)
    libXext.so.6 => /usr/lib/i386-linux-gnu/libXext.so.6 (0xf5376000)
    libQtDBus.so.4 => /usr/lib/i386-linux-gnu/libQtDBus.so.4 (0xf52e9000)
    libQtWebKit.so.4 => /usr/lib/i386-linux-gnu/libQtWebKit.so.4 (0xf2f3d000)
    libQtXml.so.4 => /usr/lib/i386-linux-gnu/libQtXml.so.4 (0xf2ef4000)
    libQtGui.so.4 => /usr/lib/i386-linux-gnu/libQtGui.so.4 (0xf238d000)
    libQtNetwork.so.4 => /usr/lib/i386-linux-gnu/libQtNetwork.so.4 (0xf2227000)
    libQtCore.so.4 => /usr/lib/i386-linux-gnu/libQtCore.so.4 (0xf1f0e000)
    libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf1ef1000)
    libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf1d78000)
    libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf1d23000)
    libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf1d06000)
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf1b4f000)
    /lib/ld-linux.so.2 (0x56555000)
    libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xf1b23000)
    libdbus-1.so.3 => /lib/i386-linux-gnu/libdbus-1.so.3 (0xf1ac7000)
    libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xf1aa8000)
    libXrender.so.1 => /usr/lib/i386-linux-gnu/libXrender.so.1 (0xf1a9c000)
    libjpeg.so.62 => /usr/lib/i386-linux-gnu/libjpeg.so.62 (0xf1a2b000)
    libpng16.so.16 => /usr/lib/i386-linux-gnu/libpng16.so.16 (0xf19f1000)
    libgstapp-1.0.so.0 => /usr/lib/i386-linux-gnu/libgstapp-1.0.so.0 (0xf19e1000)
    libgstpbutils-1.0.so.0 => /usr/lib/i386-linux-gnu/libgstpbutils-1.0.so.0 (0xf19a9000)
    libgstvideo-1.0.so.0 => /usr/lib/i386-linux-gnu/libgstvideo-1.0.so.0 (0xf191a000)
    libgstaudio-1.0.so.0 => /usr/lib/i386-linux-gnu/libgstaudio-1.0.so.0 (0xf18b0000)
    libgstbase-1.0.so.0 => /usr/lib/i386-linux-gnu/libgstbase-1.0.so.0 (0xf1839000)
    libgstreamer-1.0.so.0 => /usr/lib/i386-linux-gnu/libgstreamer-1.0.so.0 (0xf16f0000)
    libgobject-2.0.so.0 => /usr/lib/i386-linux-gnu/libgobject-2.0.so.0 (0xf1692000)
    libglib-2.0.so.0 => /lib/i386-linux-gnu/libglib-2.0.so.0 (0xf1565000)
    libsqlite3.so.0 => /usr/lib/i386-linux-gnu/libsqlite3.so.0 (0xf1450000)
    libfontconfig.so.1 => /usr/lib/i386-linux-gnu/libfontconfig.so.1 (0xf140d000)
    libQtXmlPatterns.so.4 => /usr/lib/i386-linux-gnu/libQtXmlPatterns.so.4 (0xf0faf000)
    libQtOpenGL.so.4 => /usr/lib/i386-linux-gnu/libQtOpenGL.so.4 (0xf0ea4000)
    libGL.so.1 => /usr/lib/i386-linux-gnu/libGL.so.1 (0xf0e32000)
    libaudio.so.2 => /usr/lib/i386-linux-gnu/libaudio.so.2 (0xf0e15000)
    libfreetype.so.6 => /usr/lib/i386-linux-gnu/libfreetype.so.6 (0xf0d61000)
    libSM.so.6 => /usr/lib/i386-linux-gnu/libSM.so.6 (0xf0d57000)
    libICE.so.6 => /usr/lib/i386-linux-gnu/libICE.so.6 (0xf0d3a000)
    libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xf0d34000)
    libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xf0d2d000)
    libsystemd.so.0 => /lib/i386-linux-gnu/libsystemd.so.0 (0xf0c9c000)
    libgsttag-1.0.so.0 => /usr/lib/i386-linux-gnu/libgsttag-1.0.so.0 (0xf0c5d000)
    liborc-0.4.so.0 => /usr/lib/i386-linux-gnu/liborc-0.4.so.0 (0xf0bcb000)
    libgmodule-2.0.so.0 => /usr/lib/i386-linux-gnu/libgmodule-2.0.so.0 (0xf0bc6000)
    libffi.so.6 => /usr/lib/i386-linux-gnu/libffi.so.6 (0xf0bbd000)
    libpcre.so.3 => /lib/i386-linux-gnu/libpcre.so.3 (0xf0b44000)
    libexpat.so.1 => /lib/i386-linux-gnu/libexpat.so.1 (0xf0b1a000)
    libxcb-dri3.so.0 => /usr/lib/i386-linux-gnu/libxcb-dri3.so.0 (0xf0b16000)
    libxcb-present.so.0 => /usr/lib/i386-linux-gnu/libxcb-present.so.0 (0xf0b10000)
    libxcb-randr.so.0 => /usr/lib/i386-linux-gnu/libxcb-randr.so.0 (0xf0afe000)
    libxcb-xfixes.so.0 => /usr/lib/i386-linux-gnu/libxcb-xfixes.so.0 (0xf0af5000)
    libxcb-render.so.0 => /usr/lib/i386-linux-gnu/libxcb-render.so.0 (0xf0ae6000)
    libxcb-shape.so.0 => /usr/lib/i386-linux-gnu/libxcb-shape.so.0 (0xf0ae1000)
    libxcb-sync.so.1 => /usr/lib/i386-linux-gnu/libxcb-sync.so.1 (0xf0ad9000)
    libxshmfence.so.1 => /usr/lib/i386-linux-gnu/libxshmfence.so.1 (0xf0ad6000)
    libglapi.so.0 => /usr/lib/i386-linux-gnu/libglapi.so.0 (0xf0aba000)
    libXdamage.so.1 => /usr/lib/i386-linux-gnu/libXdamage.so.1 (0xf0ab6000)
    libXfixes.so.3 => /usr/lib/i386-linux-gnu/libXfixes.so.3 (0xf0aaf000)
    libX11-xcb.so.1 => /usr/lib/i386-linux-gnu/libX11-xcb.so.1 (0xf0aac000)
    libxcb-glx.so.0 => /usr/lib/i386-linux-gnu/libxcb-glx.so.0 (0xf0a8d000)
    libxcb-dri2.so.0 => /usr/lib/i386-linux-gnu/libxcb-dri2.so.0 (0xf0a87000)
    libXxf86vm.so.1 => /usr/lib/i386-linux-gnu/libXxf86vm.so.1 (0xf0a80000)
    libdrm.so.2 => /usr/lib/i386-linux-gnu/libdrm.so.2 (0xf0a6e000)
    libXt.so.6 => /usr/lib/i386-linux-gnu/libXt.so.6 (0xf0a07000)
    libuuid.so.1 => /lib/i386-linux-gnu/libuuid.so.1 (0xf0a01000)
    libselinux.so.1 => /lib/i386-linux-gnu/libselinux.so.1 (0xf09d6000)
    liblzma.so.5 => /lib/i386-linux-gnu/liblzma.so.5 (0xf09ad000)
    libgcrypt.so.20 => /lib/i386-linux-gnu/libgcrypt.so.20 (0xf08df000)
    libgpg-error.so.0 => /lib/i386-linux-gnu/libgpg-error.so.0 (0xf08c7000)

    And finally start the client:
    ~/skype/usr/bin/skype &

  • If you find a need to get a specific file from a Linux live image, but don’t want to reboot, then there’s an easier way how to get your hands on the filesystem.

    For that mount the iso an a virtual filesystem as follows:
    sudo mkdir /media/cdrom
    sudo mount -o loop path/to/file.iso /media/cdrom
    sudo mount -o loop /media/cdrom/live/filesystem.squashfs /mnt

    Take note that the filesystem.squashfs location in the iso varies between distros, it’s easy to find the exact location with
    find /media/cdrom/ -name filesystem.squashfs

    In case you wanted to get debs for a specific package from the mounted iso:
    sudo apt-get install -s packagename | grep Inst | awk '{print $2}' > pkgreq to list required packages
    And finally run cat pkgreq | xargs sudo dpkg-repack --root=/mnt to get those deb(s).

    If you need to set a package “on hold” so it wouldn’t get automatically updated in Debian-based distributions, there are a couple of ways to do it.

    With dpkg:
    echo "packagename hold" | sudo dpkg --set-selections
    To unhold:
    echo "packagename install" | sudo dpkg --set-selections

    With apt:
    sudo apt-mark hold packagename
    To unhold:
    sudo apt-mark unhold packagename

    With aptitude:
    sudo aptitude hold packagename
    To unhold:
    sudo aptitude unhold packagename

    The following lets you determine your openssl key length and size:
    for keyfile in ~/.ssh/id_*; do ssh-keygen -l -f "${keyfile}"; done | uniq
    The output is similar to:
    8192 SHA256:eqKdP2VG3gd0pDApo4K2PyTVfY7ADoOKNE+/vLzDMz4 carry@dragon (RSA)
    Where in this case it’s a SHA-256 RSA-8192 key.

    If it happens that you committed a large file to git like I did (like an iso), in other words, made an oopsie. As you might know, just running git rm --cached bigfile would still keep it in .git history and retain the space on the disk, so we have to come up with something better that that.
    This is how to fix it:
    From the root of your repository, execute
    git filter-branch --prune-empty -d /dev/shm/scratch \
    --index-filter "git rm --cached -f --ignore-unmatch path/to/bigfile.iso" \
    --tag-name-filter cat -- --all

    where the –ignore-unmatch parameter is the file you wish to remove.
    It responds with something similar to this
    Rewrite 3f6277e3d7b938ee5191cdcc461af47ead97dcf2 (27/28) (2 seconds passed, remaining 0 predicted) rm 'Algorithms on Graphs/Week 2/Parrot-full-3.1.1_amd64.iso'
    Ref 'refs/heads/master' was rewritten
    WARNING: Ref 'refs/remotes/origin/master' is unchanged

    After that execute
    git update-ref -d refs/original/refs/heads/master
    git reflog expire --expire=now --all
    git gc --prune=now

    And all is good again.