Deploying my first web app with a backend

For the first half of my web apps class, I worked on a Twitter-like social network. It’s deployed on Heroku using Heroku’s free deployment plan. The application is implemented in Django.

Example post on Nanoblog

Example post on Nanoblog

I learned a couple incredibly useful things from this project, in addition to the benefits of the Django framework:

HTML Templating in response to AJAX Requests

This project was my first experience with HTML templating. To produce a dynamic web page, you can register a function with Django to handle a particular web request. Within that function, you can define a context dictionary of variables, such as user and posts. Then you can simply use the render(html_file, context) shortcut to produce a dynamic page. The HTML file can include template syntax like {% for post in posts %} to include the same markup for each post in a list of posts, and <a href="{% url 'user' %}"> to include a link to the profile page of a post’s creator.

Before this project, I exclusively used static HTML pages and included dynamic content using JavaScript and AJAX. AJAX, especially with JQuery, makes it easy to make “function calls” to the server: JavaScript code can make an asynchronous HTTP GET or POST request to the server, with parameters, and receive a “return value” as a JSON file in a callback function. Within the callback function, the page’s DOM can be updated with the returned data values. This typically involves JQuery code that iterates through the data and formats new DOM elements, such as table cells, lists and images.

One of the requirements for the project is that, if a user is looking at a stream of blog posts and another user adds a new post, the post should appear at the top of the first user’s page pseudo-immediately without a page refresh. The best way to implement this is to make an AJAX request to the server every few seconds asking for new posts. Before I learned about HTML templating, I would have implemented this how I just described: request new posts’ data from as JSON and use JQuery to update the page. However, such JQuery code is cumbersome and necessarily implements the same logic as the template code that is used to generate the posts on original page (assuming we want new posts to look the same as old posts.) Duplicating the DOM logic in Python and JQuery is problematic because it is bug-prone and takes twice as long to change.

Now that I’ve learned about templating, I wrote the server code to return new blog posts using the render shortcut I mentioned above. My old style would return a JSON file with data that looks like this:

{success: true, new_posts: [{id: 3, text: "hello", creator: {id: 7, name: "chris", profile_picture: "http://..."}, ...}, ...]}

The JavaScript callback would then mangle the data into DOM elements before updating the page. Using HTML templating on the server, I instead return a JSON file with all of the markup formatting done:

{success: true, new_posts: '<li><div class="blog-post"><img src="...">post by <a href="...">chris</a>: ...</div></li>...'}

The client code becomes much simpler; it just needs to insert the HTML into the right place on the page. This method also has the huge advantage of using the same HTML template that is used to generate the original page. That is, when a user first loads the page the HTML template looks something like this:

{% include 'header.html' %}
{% include 'blog_posts.html' %}

When the server responds to an AJAX request and has new blog posts to return, it returns something like:

HTTPResponse(json.dumps({success: true, new_posts: render_to_string('blog_posts.html', {posts: new_posts})}))

This method is more maintainable, less error-prone, and more modularized. I saved a lot of debugging time by keeping the code to generate the view in one place.

CSS Libraries

I got a lot of compliments on my UI but I only wrote 30 lines of CSS, and it was mainly just to position profile pictures. I did everything else by adding Materialize CSS classes to my HTML elements. On other sites I made before this project, I spent a lot of time wrestling with the quirks of CSS to try to get my elements positioned smoothly. I spent very little time on styling for this app, after I figured out Materialize’s card paradigm. Admittedly, there are significant limits on what you can do with Materialize and if I wanted to do something more complex I would need to do it manually, but for this project it was an effective way to give a minimal, uniform look and feel to my site.

Stay Mindful

A few weeks ago at the CMU iOS App Challenge, following the release of Swift and Xcode 6, Ally and I created a Mindfulness Journaling app, which you can view via the GitHub and ChallengePost links above.

Mindfulness journaling is all about non-judgmental observations of both the external world and the internal experience of thoughts and emotions. To help with this aspect of journaling, the New Entry view has, along with an empty text box, the option to create a list of observations and their intensities. For example, along with my journal entry, I could add an observation of Anxiety with the rating 7/10. As we further develop the app, this type of data can be used to track anxiety levels over time, or to track the frequency of various observations.

Stay Mindful

Web apps!

Over the last couple months I’ve been working on web applications, since I want more of my work to be accessible to more people.

Grid is a game my father, Alec Barker, originally wrote for Windows. I adapted it for web, and also added an AI API. There are three builtin AIs, and you can also upload your own JavaScript file to include your own AI. The hard AI is literally nearly impossible to beat*, partly due to the way the game is designed: the AI just follows its opponent, and neither player (usually) makes any progress. If you can think of a modification to the game that would solve this issue, let me know!

Go to App
*My sister Emily managed to beat the hard AI

*My sister Emily managed to beat the hard AI

Cubr is an application I originally wrote in Python using OpenCV for a programming course (read about that here.) As of now, Web Cubr can solve the cube, but this is only the beginning. I’m working on a more user-friendly tutorial mode, as well as webcam integration. A goal I have for this project is to make color detection adaptive. That is, every time the program detects the wrong color and the user corrects it, it will learn and become more accurate for future users.

Go to App


Reddit Scraper: Scraping top images from your favorite subreddits (with Python)

This post is a bit rushed, but if I don’t post it now, I know I won’t have a chance for another few weeks. I leave for the airport to go back to CMU tomorrow. My bags are packed, everything’s all squared away… except I want to write about a little project I’ve been working on this summer. So, without further ado…

The problem

If you don’t already know, Reddit is a hub of internet links and associated comment sections. The top links shown are the ones that receive the most upvotes (user-submitted clicks of approval.) Reddit is divided into “subreddits”, each with its own community theme. For example, is a community where people share links about the tv show Dexter. /r/wallpapers is a subreddit where people post links to their favorite desktop wallpaper images. I like to look at /r/aww when I need some cheering up. People post links to pictures of adorable kittens and puppies and all sorts of cute things.

But I’ve been thinking, wouldn’t it be great if, instead of navigating to Reddit to see those pictures, I just had to look at a folder on my computer? Or even just at my desktop wallpaper? And what if there were a different adorable kitten on my desktop every time I looked?

The solution

I created an app that can grab the top pictures from any subreddits and save them to any directories on your computer. Mac OS X has the option to change the desktop wallpaper, at a set interval, to a random image from a given directory. So, I configured Reddit Scraper to grab the top 5 pictures from /r/aww every day at 8:00 PM (using cron) and put them in my wallpapers directory.

The app is very configurable. You can take images from any number of subreddits and save them in whichever directories you choose. If you want to, you can even take non-image files, like HTML files, and save them for offline viewing. See the “How to use it” section below for details on how to start getting daily (or even hourly, or minutely, or weekly…) updated wallpapers or archival folders from /r/wallpapers, /r/earthporn, you name it!

The results

The configuration GUI

A folder I’ve scraped photos into

My (current) desktop, thanks to /r/wallpapers

How to use it:

1. Fork the source on GitHub (link at the top of this post)
2. Make sure you have Python 2.7 and other dependencies installed (notably PRAW)
3. Run and use the GUI to configure your settings (make sure to press Save)
4. Use your favorite process scheduler to run at whatever interval you choose
5. Sit back and let reddit_scrape do all the work.

That’s it! If just one person gets use out of this, I’ll be happy. Please comment below with any feedback (suggestions, bugs, etc.)

Connecting iCloud PhotoStream to Dropbox

If you live in the world of Apple, you probably know what the Photo Stream is. You take a picture on your iOS device, and it automatically goes into your private photo stream. You can also choose to share it with friends or family in a photo stream to which they’ve subscribed. In my family, each of the four of us has our own photo stream, and we use this to share photos with each other. It’s been a really nice way to stay in touch with my family in Seattle when I’m in Pittsburgh. Yes, of course we talk on the phone too, but the photo stream allows us to share small things right then and there.

ANYWAY, this isn’t an advertisement for the photo stream.

If you have a Mac with iPhoto, you may have noticed that your streams, both private and shared, are synced with iPhoto as well. Cool! But what if you like keeping your photos somewhere else, like Dropbox or Google+? It turns out, iLife stores your photos in a really strange way. Each photo is in its own directory, and if you subscribe to multiple photo streams, your photos will have duplicate names. So, getting photos from your stream to another place on your system isn’t as easy as dragging and dropping.

That’s why I wrote this script, which recursively moves all the photos from the (hidden) photo stream storage location into a specified folder and guarantees unique filenames. On my computer, I set up a cron job so all the photos from my stream are synced with Dropbox every evening.

This has several advantages: one, since it’s a shared Dropbox folder, my family members who use Windows (all of them) can access the stream photos from their computers, without relying on a fast connection from Two, it means I have all my photos in one place, in Dropbox, rather than scattered between iPhoto and my file system. iPhoto is great if it’s all you use, but if you need integration between your photos and the rest of your OS, iPhoto doesn’t like to play nice.

So, now that we’ve covered the rationale, here’s the script:

Exploring Python’s new Enum type

The first alpha for Python 3.4 was released yesterday [link]. Two of the major changes to 3.4 were new additions to the standard library: functools.singledispatch and enum.Enum.

Enumerations (Enums) are a way to define a finite set of cases or states with no programmatic ordering. For example, we could have an Enum class with members [“apple”, “banana”, and “orange”], but it wouldn’t make sense to have an Enum class with members [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] because these have a programatic ordering and are better expressed as integers. Enums have been around in other languages (like C#) for years. There are many times when it’s useful to have a variable representing a particular state, and Python developers have had to find ways around it. Tkinter, for example, has N, S, W, E as cardinal directions used for anchoring widgets. These are stored as integers.

Up until Python 3.4, there were two prominent ways of implementing Enums: global constants, and class variables. Module-level enumeration states are almost always stored as globals. For example:

>>> RED, GREEN, BLUE = range(3)
>>> ORANGE, BANANA, APPLE = range(3)

This works well enough, although I’ve intentionally provided an example where it causes problems. Is ORANGE a color or a fruit?

>>> color = RED
>>> color == RED
>>> color == ORANGE
>>> color + GREEN == BANANA
>>> # Completely nonsensical

Another option is to use classes to get around this:

>>> class Color(object):
        red = 1
        green = 2
        blue = 3

>>> class Fruit(object):
        orange = 1
        banana = 2
        apple = 3

This sort of deal with the ambiguity of the word orange:

>>> color =
>>> color == # no one would make this comparison

So at least now the members are assigned to a class rather than just global integers. But comparisons between makeshift enum classes still give undesirable results, and we can still perform integer operations on them.

Enter the enum module.

>>> # The new way of doing things:
>>> from enum import Enum
>>> Color = Enum("Color", "red green blue orange black")

>>> +
Traceback (most recent call last):
  File "<pyshell#16>", line 1, in +
TypeError: unsupported operand type(s) for +: 'Color' and 'Color'
>>> # That makes a lot more sense!

This is a huge improvement over the old way of implementing Enums. We can’t add enum members – not without defining a special __add__ method for our class, anyway! However, this is certainly possible, if you want to define a custom __add__ so that plus is Color.purple. Also, won’t evaluate as equal to anything but itself, regardless of what its internal integer value is.

Wait, its what? Now that we have a dedicated Enum class, why do we need integer values associated with enumeration members? The main reason is for backwards compatibility. Some code relies on modules’ integer values their enum members. In fact, the enum module also provides an IntEnum class, where members can be treated as ints, specifically to support backwards compatibility. Another benefit of having integer values associated with enum members is the ability to include aliases, such as aubergine and eggplant in this example:

>>> class Food(Enum):
        apple = 1
        banana = 2
        orange = 3
        aubergine = 4
        eggplant = 4  # Synonyms can be aliases
        def eat(self):
            print "nom!"

>>> ==
>>> ==
>>> ==
>>> ==
>>> for food in Food:
>>> type(Food.eggplant)
<enum 'Food'>
>>> isinstance(Food.eggplant, int)

This offers a much more intuitive way to deal with Enums than we have in python 2.7.


Python 3.4’s Enum implementation is definitely better than what we have been using. However, it still relies on externally associating enumeration members with integers. This is mainly because the Enum class is long overdue, and there have been so many workarounds implemented. Also, there’s no reason the enum module can’t be ported to Python 2.7, other than that the core developers really want us to migrate to 3K. I don’t blame them – it has a lot to offer. But there just isn’t enough traction yet, and it may be worth it to include Enum in the Python 2.7 standard library. Or perhaps as a 3rd party module… Thoughts?

Update: Redditor eliben commented that there is an official backport for 2.4-3.3.