Posted on May 28, 2009

Premature Optimization Is the Root of All Evil

…so says the famous quote, my current mantra.  This leads to my primary mission (or possibly, it would be more appropriate to call this my Prime Directive) is to code and get it all out. Brainstorm in code quicker and faster.

I have this extremely bad habit of writing 10 lines of code before shelling out the rest of the day learning how to improve these lines in R.  Yes, I learn a lot through the process, but graduate students are responsible for more than just learning. Sometimes, there are those that actually have no interest in what I have learned but only my results.

This idea is nothing new to me, but this post is an attempt to enforce a turning of a new leaf. It was ridiculous for me to waste the previous hour intending to optimize TWO lines of code! I cannot and will not do this anymore. Those two lines of code were not important enough to waste that much time.  Before my bedtime, my whiteboard may be plastered with a repetition of “I will refactor my code later and not now.”  Unless the code is being published, procrastinated optimization is far better than procrastinated results.

Am I alone in this endeavor?

Posted on Mar 14, 2009

A Little Bit About Data Frames in R

Data frames in R are much like DataSets in SAS, SPSS, .NET, etc. Really, they are just spreadsheets that feel like a matrices. We can use these to look at numerical data along with any meta data or characteristics associated with the numbers though numbers are not required. From the R Documentation, “a data frame is a list of variables of the same length with unique row names”, and also it is “a matrix-like structure whose columns may be of differing types (numeric, logical, factor and character and so on)”.

Let’s take a look at an example. First, we start with generating a 3 x 3 identity matrix and assigning the matrix to the variable, mat.

[code]
mat = diag( 3 )
[/code]

By typing mat, we can see the output.

     [,1] [,2] [,3]
[1,]    1    0    0
[2,]    0    1    0
[3,]    0    0    1

Next, we are going to convert this matrix to a data frame called mat_dataframe and output it.

[code] mat_dataframe = data.frame( mat )[/code]

  X1 X2 X3
1  1  0  0
2  0  1  0
3  0  0  1

Notice that the column names are X1, X2, and X3 and that the row names are 1, 2, and 3. Say we want to add more columns and rows to our data frame. Let’s first start by appending a row to the “mat_dataframe.”  We do this with rbind.

[code]
mat_dataframe = rbind(mat_dataframe, c(2,2,2))
[/code]

We have added a vector of twos to the next row of the data frame.  Here’s what mat_dataframe looks like so far.

  X1 X2 X3
1  1  0  0
2  0  1  0
3  0  0  1
4  2  2  2

Now, we should try appending 2 columns to the mat_dataframe using 2 different methods. The first line will create a new data frame from the original data frame and append a column called “City” with “Dallas” as the entry for each row. The second takes this data frame and adds another column called Color with entries blue and green.

[code]
mat_dataframe = data.frame( mat_dataframe, City="Dallas" )
mat_dataframe = cbind( mat_dataframe, Color=c( "blue", "green" ) )
[/code]

Now, the mat_dataframe looks like this.

  X1 X2 X3   City Color
1  1  0  0 Dallas  blue
2  0  1  0 Dallas green
3  0  0  1 Dallas  blue
4  2  2  2 Dallas green

Notice that once blue and green were both used, they were both repeated. Before we move on, let me mention a gotcha when adding columns. On the City column, I simply inserted Dallas for each row, but under the Color column, I added 2 different colors. What happens if we specify three values? Let’s try this with a new column called Country.

[code]
mat_dataframe = data.frame( mat_dataframe, Country=c( "USA", "Canada", "Mexico" ) )
[/code]

We get the following error…

Error in data.frame(mat_dataframe, Country = c(“USA”, “Canada”, “Mexico”)) : arguments imply differing number of rows: 4, 3

A rule of thumb: make sure the number of values being assigned divides into the number of rows (or columns) of the data frame. If our data frame had 6 rows (or 9 or 12 or … ), we could have used the above code.

Our data frame is essentially a matrix with a couple of attached column vectors containing strings. This may not seem very useful at first, but it is a wonderful data structure, making some statistical methods among other things easier to use. Soon, I will post a basic ANOVA example using data frames.

Posted on Aug 24, 2008

Project Euler

Anthony turned me onto this new project called Project Euler.  On the site, there are currently 202 computational math problems ranging from easy to extremely difficult.  Several of the problems can be brute-forced in order to find a solution, but there are some that are just not possible to solve in this manner in this lifetime, and thus, clever methods are needed to solve the more difficult problems.  As you may guess, I am becoming obsessed with this site; last night, I stayed up until 6:30am working on as many as I could.  So far, I have solved 14 of them.

I was not too savvy with Python until now that I am restricting myself to using one and only one programming language for these problems; I have learned much about the language, and I’m quite impressed.  It is always great to have mundane aspects of coding shot into oblivion because they really just are not needed anymore: well, maybe some will continue to believe C is necessary for all.

In the next couple of days, I am going to start uploading my code for these solutions; if anyone cares, maybe it will be a learning experience for us all.  For the most part, I have the math know-how to solve these problems within the recommended minute, but I ask that any and all advice be given to improve my Python skills.