Comments on: data.frames in Python - DataMatrix

By: Einar

Einar — Fri, 07 Nov 2008 22:09:33 +0000

The last line should be
datamatrix[col_names[int(num)-1] = col_data

and to save, the writeMatrix function is not part of the class, so assuming you have imported “datamatrix”:

datamatrix.writeMatrix(datamatrix_object,open(”somefile.txt”,”w”,header=true)

Notice that this will work only with the git version of DataMatrix (since it has changed a bit internally)

With the old version, I think you should do

datamatrix.writeMatrix(datamatrix_object,”somefile.txt”)

Where datamatrix_object is your DataMatrix instance.

By: Mike

Mike — Fri, 07 Nov 2008 21:49:12 +0000

Hi Einar,

Thanks for the reply. I am new in python so I am not so sure how you said works.

Here is part of the code I wrote:
===========================
for num in column:
(indent) col_data=data_obj.getColumn(col_names[int(num)-1])
(indent)(indent) for i in range(1,len(col_data)):
(indent)(indent)(indent) col_data[i]=col_data[i].replace(old,new)
(indent)(indent) datamatrix(col_names[int(num)-1])=col_data
===========================
NOTE:
(indent)-> represents indentation since the blog does not support it
column-> a list of column which I would like to apply the substitution
col_names-> a list of column names
data_obj-> datamatrix object
old-> some pattern I do not want to keep
new-> substitute old by the new pattern

The last line of the code, which I mimic what you said about reassign it with datamatrix[columnname] = list, does not work since it assigns to function call.

I think my question is related to how to make changes within column entries and then save and write the datamatrix object into a file.

I really appreciate any comments and suggestions you gave.

Mike

By: Einar

Einar — Fri, 07 Nov 2008 19:58:25 +0000

DataMatrix entries are lists, hence mutable. The latest version (in git) supports __setitem__ so you could make a list of tuples in the same format as the entries you read (e.g. a list of (rowname, value) tuples) and reassign it with datamatrix[columnname] = list .
Then you can use the writeMatrix function (use the one in the git repository, the one in the 0.5 version is broken as far as I remember) to write the modified DataMatrix to a file.
I have made loads of enhancements with respect to this version, so I guess I should put a new version out soon.

By: Mike

Mike — Fri, 07 Nov 2008 19:19:23 +0000

Hi,

I found your module quite useful! really like it.
However, one thing I dont get is how I am able to modify some entries in a few columns and then write the whole matrix back to a file with the modification I made. Say, I have a file with 10 columns and each column has a few thousand entries. If I would like to rename entries, whose value <10, as “NA” for only some columns, I do not see a way to write the modification into a file. Can you give me some examples in this purpose?

Thanks!

By: Mike

Mike — Fri, 07 Nov 2008 19:19:05 +0000

Hi,

I found your module quite usefule! really like it.
However, one thing I dont get is how I am able to modify some entries in a few columns and then write the whole matrix back to a file with the modification I made. Say, I have a file with 10 columns and each column has a few thousand entries. If I would like to rename entries, whose value <10, as “NA” for only some columns, I do not see a way to write the modification into a file. Can you give me some examples in this purpose?

Thanks!

By: Einar

Einar — Sun, 29 Jun 2008 15:32:47 +0000

Good catch. I found the scipy web page a little disorganized, that is why I probably did not find it. I will see if I can integrate the file management part (AFAIK numpy provides only simplistic ways of converting files into arrays) with either recarrays or Record Arrays.
Thanks a lot for the suggestion!
EDIT: I have briefly looked at numpy and looks like you can’t really add arrays in an iteration like my module does. The only alternative would be to use fromfile to load the file directly and add the record array on the spot, but I’m not sure if I should use that since numpy’s documentation says that the function is not very robust.

By: Tony

Tony — Sun, 29 Jun 2008 15:22:52 +0000

Nice work.

Have you played around with numpy? This is pretty similar to Record Arrays [http://www.scipy.org/RecordArrays] and recarrays (which, is similar to Record Arrays, but slightly different) in numpy.