python

Stumbling through or-tools

The other day I pip installed google's or-tools and started madly optimizing every facet of my life, using constraint programming. Er, well, actually I've been futzing about solving toy problems and trying to figure out how to use or-tools?.

tags: python, or-tools, constraint programming,

Transit Effectiveness redux

In a previous post I plotted some transit statistics and waved my hands around at the apparent and obvious correlation without doing any math. I figured I should re-address this and do some minimal statistics (what? I'm lazy) to show that there is some relation between transit usage and how easy it is to get around by transit (a shocking assertion). Part of the reason I didn't bother with any deeper analysis last time was that I figured my claim was obvious. But, on reflection, there is a lot going on there that could be analyzed to death, and since I have nothing to do this afternoon...

tags: transit, open data, python,

Abusing Chi Squared

This is one of those questions I could probably answer with research, but I'm lazy so I am going to do simulations. Anyways, it came up in my life to check a data-set to see if the values are normally distributed. There are a couple of ways of doing this (I lean towards doing a KS test) but one that was recommended was to do a $\chi^2$ test. Of course the $\chi^2$ test typically requires the data to be in discrete bins, and this got me thinking: surely the test itself is highly dependent upon the bin size I choose so, presumably, I could fiddle with that variable to get whatever answer I wanted. Presumably.

tags: python, statistics, random numbers,

The 2016 Census and Transit in Edmonton

Recently I've been posting about Edmonton Transit and drawing a lot of my data from the 2014 Edmonton Census. Well the 2016 census data is now available on the Edmonton Open Data Portal so I should take a peak and see what's changed.

tags: python, mapping, open data, transit,

Working around a mapping bug in cartopy

I recently discovered a bug in how I was generating choropleth maps with cartopy. Basically the add_geometries() function for adding shapes to matplotlib maps was not mapping the correct face colours to the correct geometry.

tags: python, mapping, cartopy,

Transit Effectiveness as a Predictor of Ridership

Yesterday I put together some maps showing some results culled from google maps on how effective Edmonton Transit is vs driving in your own car. It looked pretty grim for the 'burbs, with average transit times being 20-30min longer than the equivalent trip by car (almost twice as long!). But I didn't really answer whether or not this any impact on actual transit ridership. Intuitively we think it should, but there are lots of other factors as well, such as economics. If you got no money you're still to take the bus (because it is cheaper than driving, slightly) even if it takes you hours.

tags: transit, open data, python,

The Effectiveness of Edmonton Transit

I was having a conversation, the other day, about how much of a pain it is to take ETS to and from the 'burbs. I am a big fan of not driving as much as possible and I resisted owning and driving a car for years in Edmonton (notably when I lived downtown and either worked downtown or at the University) but no longer, I have a car. My particular breaking point was working in a business park that wasn't really transit accessible -- by bus, train, bus, and then walking my trip to work took over an hour each way, with a car that dropped to 20 minutes max. I figure this experience generalizes well and exlains why transit ridership is really low in Edmonton. Transit takes forever and it sucks, whereas everywhere is a 20-30min drive from everywhere else in this town.

tags: transit, open data, python, mapping,

Finding the hot singles in my neighbourhood

I've been working on a project for the past few weeks that involves parsing a bunch of data sets to generate some aggregate statistics at the neighbourhood level in my hometown of Edmonton. Staring at tables of numbers and scrutinizing a ROC curve can only inspire you so much. Today I'm taking a break and making some maps of Edmonton's latest property values dataset and most recent city census (done in 2014).

tags: mapping, python, edmonton, open data,

Building my own spam bots

A few days ago I decided to play around with Markov chains, to improve the playlists generated on my computer actually, but somehow I got sidetracked and ended up making a horrible twitter bot
trigram 6

tags: python, javascript, markov chains, spam,

Partitioning Teams Part 2: Electric Boogaloo

In a previous post I looked at various ways of partitioning up a group of people into teams, such that each individual's preferences of teammates is taken into consideration and the overall happiness of the team, and thus the corporation, is maximized. I've been spinning ever more complicated way of doing this in my head, so why not try one out?

tags: partitioning, python,

Dumb ideas and experiments

Recently I was talking, on the internet no less, with someone who was trying to take some data presumably drawn at random and find the parameters of the corresponding pdf (some weird Weibull something-er-other for doing risk modelling but that's beside the point for what follows). In simple cases the common man would look up the maximum likelihood estimators for the parameters of the given pdf, or maybe numerically find the parameters that minimize the Kolmogorov-Smirnov statistic or something. This guy's plan was to do a non-linear least squares fit of the pdf.

tags: random numbers, python, statistics,

Partitioning Teams

I found an interesting problem in Ask Metafilter the other day about sorting people into work groups. The problem is this: suppose you have a group of 90 people and you want to sort them into 3 equal sized teams. To facilitate this you ask them to write down the 5 people they most want to work with. What method (algorithm) should you use to sort these people such that you accomodate the most preferences?

tags: partitioning, python,

The toy production line, revisited

I built a toy model of a production line, earlier, and now I'm back to bludgeon it with more statistics.

tags: production, stochastic process, python,

Guessing random numbers and numberphile

A recent numberphile episode had a neat idea of how to, apparently, beat the odds in guessing whether one known number is larger than an unknown random number, by adding a third random number.

tags: random numbers, numberphile, python,

A match-stick production line

I am reading The Goal and in one chapter there is a toy model of a factory, in which boy scouts move matches from one bowl to the next trying to move matches to the end of the line, how many they can move from their bowl determined by a roll of a dice.

tags: production, stochastic process, python,

Was it the PC party's election to lose?

I've been hearing a lot that this past election was the PC party's to lose, and that what we saw was not an "Orange Crush" as much as an ABC -- Anything But Conservative.

tags: elections, python, mapping,

The closeness of the 2015 Alberta election

I was thinking, while I drove home today, about how close so many of the ridings seemed, and whether or not voter turn out was correlated to how contested a riding was.

tags: elections, python, pandas,

Vote Splitting in Alberta

Alberta's big election was last night, and an orange wave overcame the province resulting in an NDP majority. This ends the 44 year reign of the Progressive Conservative party, and marks the fourth change in government in Alberta, ever, in 110 years. Yes only 4.

Anyways. For a long time …

tags: Elections, python, pandas,

Storage tank deflection part 2

Continuing on from last time I looked at storage tanks I want to look at how well the one foot method lines up with the equation I derived for a simple cylindrical shell.

tags: API 650, hoop stress, python,

Metrics and the Illusion of Progress

This isn't really engineering related, directly, but I think it is an interesting example of when data can lead you to the wrong conclusions.

Where I work we have a central document management database that manages the flow of documents from creation through approval and release. So, for example, I …

tags: quality system, metrics, missing data, python,


Page 1 / 2