textio¶

Examples¶

Using text_data objects

Using ioconfig objects

Code Help¶

read_DS3505(filepath, has_headers=True)[source]¶

Text reader for DS3505 data (space delimited) with some fixes

Weather data downloaded from the following website has a peculiarity [http://gis.ncdc.noaa.gov/map/viewer/#app=cdo&cfg=cdo&theme=hourly&layers=1&node=gi] in that it has some upercase T’s that are rarely needed, but ruin otherwise uniform space formatting.

Parameters:	filepath – filepath to DS3505 data has_headers – set False if filepath does not have headers. This doesn’t seem to ever happen for DS3505 data.
Return tdo:	returns a text data object with DS3505 data in it.

class text_data(headers=None, row_data=None)[source]¶

A text data object is a very simple template structure for passing text type data (usually lists of weather or climate data entries) around between functions.

_build_col_data()[source]¶

Builds column wise data dictionary structure out of row_data. (row_data is a list of rows, where each row is a list, so row_data is a list of lists). Col data is a single dictionary, where the keys are the “headers” and the value is the list of all values in that column from the top down.

_build_row_data()[source]¶

Builds row wise data from existing column data. The opposite of _build_col_data.

static _enf_unique_headers(headers)[source]¶

Appends digits to duplicate items in a list. Used to ensure each header is a unique string so a column-wise dictionary can be built.

For example, a text file with headers ["name", "tag", "tag", "tag"] will be changed to have headers ["name", "tag1", "tag2", "tag3"].

Parameters:	headers – list of string elements intended for use as headers.

read_csv(text_filepath, delim=', ', has_headers=True)[source]¶

simple default reader of a delimited file. Does not read fixed-width

Parameters:	text_filepath – csv filepath to read from delim – delimiter to use, defaults to comma has_headers – Set “False” if csv file has no headers (this is bad, you should give your file headers)

read_json(json_filepath, row_wise=None, col_wise=None)[source]¶

Reads the contents of this tdo from a json file created by the text_data.write_json() function. Please note that this text_data class is designed for use with tabular type data, so this should function will not read ALL json files in a satisfactory manner. Users wishing to read json files in a general sense should simply use the json module and invoke json.loads and json.dumps directly on their data.

Parameters:	json_filepath – json filepath to read from row_wise – read json file objects in as rows col_wise – read json file objects in as columns

write_csv(text_filepath, delim=', ')[source]¶

writes the contents of this text file object as a CSV

Parameters:	text_filepath – output filepath to write csv .txt delim – delimiter to use. defaults to comma

write_json(json_filepath, row_wise=None, col_wise=None)[source]¶

Writes the contents of this text data object to a json file. Note that json format does not support complex numbers with imaginary components. Also note that json formats are dictionary like, in that they preserve relationships, but do not display the list of relationships in any particular order.

Parameters:	json_filepath – output filepath to write json row_wise – set to TRUE to save each row as its own structure col_wise – set to True to save each col as its own structure

class ioconfig(input_filepath=None)[source]¶

An ioconfig object is an extension to the text_data_class it has the same methods of text_data_class, plus an add_param function and simplified write(), and read() methods for making visually formatted csv files so users can edit config files directly.

It is used to generate “config” files, lists of important inputs to a set of complex functions. Allows you to save inputs and call them over and over again to avoid tedious input definitions, also facilitates good record keeping by keeping a hard file with a summary of all inputs alongside any outputs that might have been generated.

When a param has been added or imported, its value may be accessed with

ioconfig_object['param_name']

Parameters:	input_filepath – Optional filepath to read/write this ioconfig object to/from. If you do not provide a filepath here, you can provide one when invoking the read() and write() methods.

_interp(in_type, in_value)[source]¶

allows interpretation of config values based on type

add_param(param_names, param_values)[source]¶

Adds parameters to the ioconfig object

Parameters:	param_names – A string, the name of the parameter param_values – the value of the parameter. May be any built in datatype. (bool, int, str, list, long, etc)

when a param has been added or imported, its value may be accessed with

ioconfig_object['param_name']

read(filepath)[source]¶

Reads the contents of a csv file generated by ioconfig.write(), interprets and formats each of the variables to the state it was before exporting. Once finished, this the ioconfig object will have all its standard attributes including conf_dict.

Parameters:	filepath – filepath to a csv or txt file generated by the ioconfig.write method.

write(filepath)[source]¶

Prepares the row data with nice visual formatting then calls the text_data.write_csv method.

Parameters:	filepath – filepath to write csv to. (.txt, .csv, etc)