Viewable prototype

I finally feel comfortable showing my app. It's here: moretouching.com/parking

Currently the radio buttons control the day and time. Loading the page sets the correct values by the current time. Wanted to add new buttons to distinguish between no parking versus parking allowed. I tried doing this by the SQL query filtering, with something like this:

other stuff.. AND (no_parking='true' OR no_standing='true' OR no_stopping='true').

It wouldn't work. I spent about a day trying to figure out what was wrong. Turns out, Google Fusion Tables don't support the OR operator. That would've been useful information, anywhere in their documentation. I had to find out by looking at some Stack Overflow posts. (maybe there's a way to work around it using IN?)

As a work around I'm adding a new field called noParkingAll. Kind of clunky, but it's late and I'm distracted. Also makes me regret the no_parking style I went with earlier.

On a different note, I found some examples of using different colored dots for different types of points. Certainly something I'll want to do, with green and red dots. Maybe some orange and yellow in the future.

Data precision vastly improved

Since my last post I found many instances of  incorrect data. I won't go into too much detail, but here are the highlights:

1. Mouse over fusion tips seemed to be showing or causing incorrect data points to be displayed, disagreeing with the radio buttons. I turned them off for now, and instead rely on the pop up bubbles for information.
1. Turns out you can select the ordering of the columns, which effectively determines what comes up in the pop up bubble. There must be a way to select which fields to show here, but for now this works fine.
2. Some data points were showing incorrect data. It turned out to be an issue with how I was calling the parser. I didn't realize some parking descriptions had multiple sentences. The parser was treating the subsequent sentences of a description as the next description, so the data became misaligned with the descriptions after this point. Fixed by adding a new annotator option when calling the parser.
1. I initially thought it was related to times that were of the "11pm to 3am' flavor. It wasn't, but my code wasn't handling this case anyway, so I added it in.

At this point all the days and times seem to be displaying correctly corresponding to the descriptions.

What I'd like to do now is determine whether a sign says you can park somewhere, or it says you can't. I can display the differences via red and green dots. 

I'm going on my vacation soon. I want to keep working on this but I don't know if I'll be able to. Perhaps I shouldn't, even if I could. Debating taking my laptop with me on this trip.

Some hopeful developments

There have been a number of developments since I last wrote.

1. I successfully loaded all the points to the fusion table. It was much smaller as a CSV.
2. I modified the javascript so only points that fit the exact time criteria were shown, with appropriate radio buttons to control it.

For verification I wanted to look at the description of the points, but because I have so many fields, I couldn't see the description in the pop up bubble. In my attempts to figure out how to show only what I wanted in them, I decided to try deploying one of Derek Eder's existing projects called 2nd City Zoning. This process became a huge mess, but today I finally figured everything out. I'll try to outline everything I did here, in case anyone needs to replicate this.

• The instructions in his README seem simple, but to get it to work requires a lot of setup.
• A lot of it had to do with setting up the right Ruby things. This site provided most of what was needed. Some highlights:
• Install Homebrew, which in turn allows easy installation of other things.
• Install rbenv, which allows for custom ruby environments. Instead of using the built in OSX ruby, we can choose which ruby version to use, and easily switch between different ones.
• Ruby Gems is configured to install things to the local user directory.
• I created environment variables GEM_HOME and GEM_PATH, both set to a ".gem" directory in my home folder.
• Had a hell of a time figuring out why I was having problems. Turns out it was because I had quotes around the path names, like this: GEM_HOME="~/.gem".
• The correct way is shown below.
• GEM_HOME=~/.gem
• GEM_PATH=~/.gem
• I also installed Bundler, which takes care of installing all the dependencies.
• Seems to work with the "Gemfile" files.
• Still, I had trouble running jekyll, which complained about a dependency that I could see was installed.
• The answer was to execute the command using bundle, since I was managing the packages with it.
• The instructions also talks about adding a line to my bash profile, which perhaps removes the necessity of doing this. I don't know, haven't tried.

Now that I have it running I want to start playing with the javascript, so I can see how to modify what shows up in the pop up bubble. Then, I need to display the description.

I have a sneaking suspicion that the points aren't labeled properly because it doesn't pass the eye test. There seem to be too many no parking areas at night. Maybe this is actually correct, but I want to be sure.

---

I went to a hackathon on Sunday, and it was time very well spent. I got to meet LA's chief data person Abhi, who was hired by Mayor Garcetti to open up a treasure trove of LA's data. I asked him about the parking signs, and unfortunately he said that data doesn't exist in consolidated form. That's fine, I had a feeling. Still, I just know that it exists in some form, probably scattered, simply because of the existence of parking enforcement. How could they do their jobs without this information being available some how? We just need to get the data from them, and put it together.

There's a project competition due in September. I'm going to submit my project for it. The winners will get funded for 4 months, which sounds awesome. If I can't get the data I need by then, I might manually walk around a bunch of city blocks and record the data. Then show my project using this data, as a proof of concept. Hopefully it is compelling enough.

Fusion tables

I was having some luck getting fusion tables to work using Derek Ender's stuff. I added radio buttons to distinguish between days of the week. I wanted to add the same thing for times of day. A way to accomplish this is to have a fields corresponding to every time slot. Of course, this could greatly increase the size of the data. I wasn't sure if it would break fusion tables, nor did I know exactly how much, but I decided to go for it.

The resulting JSON turned out to be 1.23 GBs. Too big. The total size allowed is 1 GB, and each table can be up to 250 MB.

I was originally setting the fields as 'true' or 'false'. Changed it to 't' and 'f' in an effort to limit size.
---
Reduced it down to 1.08 GB... I mean, it seems small, but 0.15 GB is still 150 MB. Nothing to scoff at.

I could try reducing the field names too... But no matter what I do, it'll be tough to get the table size below 250 MB.

The original JSON with most fields cut was 142.7 MB, while the CSV was only 54.1 MB. That means the CSV was 38% of the original JSON size. We'd still be too big. I'd need to get the JSON down to 658 MB for the same math to work out for the 250 table size limit. Maybe try splitting the data by day?

Depression

http://www.wsj.com/articles/entrepreneur-finds-profitable-niche-parking-spaces-1414965251

So here's the guy who made Smooth Parking.

I've looked at the page's source code, and I think I know how he did it. I'm impressed he did all that honestly.

I'm very upset. At this moment I'm trying to build something that already exists, and is very good. Granted, my ultimate goal is to bring this to Los Angeles. However, I felt like I needed this NYC version to have some success to convince the city of LA to help me with this endeavor. Now, I feel that I have no shot at it. How can I compete with an already existing, well functioning, multi platform (both Android and iOS) version of something I haven't even made yet?

I feel like I've thrown away the past six months on this endeavor.

Some of the smallest silver linings:

The app doesn't seem popular. Why? I'd use it.
The aforementioned article was written in Nov 2014. He registered his domain in 2011. According to his LinkedIn, he started the company in mid 2012. It tells me that its taken years to get this thing off the ground.

---
I went for a walk. Had a lot of thoughts in my head. Also took a bit of time off to play some Hearthstone. I feel much better. I think I can make a pretty good web interface. Let's start there.

Prior art discouragement

I started finding existing products that do a lot of, or even do entirely what I want to do.

1. Street Parker has a nice interface. It doesn't show exactly what I want, but it has all the tools to make it happen. It also uses some open source (?) software for the interface. I want to look into that.

2. Smooth Parking is basically what I want, and it already exists! Dammit. They already have a nice web interface and mobile apps. 

3. An app that used to exist called PrimoSpot seems like it did a decent job at presenting parking info. I think I had read about it but for some reason the app is gone.

Where do I go from here? Do I continue this journey? To get to even Smooth Parking's level of features and polish will take me a very long time. I'm very discouraged.

I did notice that Smooth Parking's web interface treats times like I do - in 15 minute increments. I guess I wasn't completely crazy when I picked that. And the web interface is actually fairly slow when zoomed out. Plus, the mobile app doesn't show data until you're zoomed in up to a certain level, presumably because of performance.

I won't be discouraged. I think there is still hope.

Fixing errors

My logic for comparing start times and end times was funky. I cleaned it up to be much simpler, and now the times seem to match up correctly with the file names.

I noticed that "except Sunday"s were not working. I'd look at a map of Sunday, and see a bunch of parking signs that were supposed to invalid on that day. I looked at some code but it isn't obvious what the problem is.

Starting writing unit tests to address both these problems. They're great - they provide a nice amount of reassurance that the code works. They're also great for debugging, especially when tweaking regular expressions to get them just right.

Something seems up with getting the "except" regex. Seems like it should be obvious, since the simplest case of "except sunday" doesn't pass. Could be in the actual function, could be in the regex, could be a mistake in calling the wrong thing somewhere (wires crossed).

UPDATE

Figured out the "except" regex error. Now I'm re running the pipeline, and starting all the way back at "skipParse". I need all those intermediary steps which take care of getting the valid days.