Supported Data Files

PerceptiLabs' Data Component currently supports the following file formats: .csv .txt .npy (NumPy file) .npz (NumPy zipped file). The Data Component can also be configured to point to a directory of image files in the following formats: .jpg .png .jpeg .tiff .tif. The files are read by PerceptiLabs in alphabetical/numerical order.

Note: Paths cannot contain non-ascii characters.

Comma Separated Value (.csv and .txt) Files

PerceptiLabs' Data Component can import data from a .csv file, a popular text-based format, where data values are separated into "columns" or "fields" by commas, and each line represents a "row" of data. Files with the .txt extension are also treated as .csv files. PerceptiLabs supports UTF-8 unicode files.

The first line of the file must contain strings which represent the "column" labels, and the values in each subsequent row must be numeric. Note that PerceptiLabs will convert all integer values to floating point internally.

The following is an example of a valid .csv file:

Wine,Alcohol,Malic.acid,Ash,Acl,Mg,Phenols,Flavanoids,Nonflavanoid.phenols,Proanth,Color.int,Hue,OD,Proline
1,14.23,1.71,2.43,15.6,127,2.8,3.06,.28,2.29,5.64,1.04,3.92,1065
1,13.2,1.78,2.14,11.2,100,2.65,2.76,.26,1.28,4.38,1.05,3.4,1050
1,13.16,2.36,2.67,18.6,101,2.8,3.24,.3,2.81,5.68,1.03,3.17,1185

After selecting a .csv or .txt file in a Data Component, you have the option to select which fields you want to include in the dataset:

Click the fields dropdown (1) and then click on the individual fields listed (2) to toggle whether they should be included, or click on Select All (3) to include all fields. Once you're done, collapse the dropdown (3) to apply your selection(s).

When selecting and applying the fields, you can also select the Shuffle option (enabled by default). This shuffles the dataset so that the order is different every epoch, which can be an effective technique for teaching a model to generalize better.

NumPy (.npy and .npz) Files

PerceptiLabs' Data Component can import data from saved from NumPy, a Python framework for working with arrays of data.

Using NumPy's numpy.save method, you first export/save a NumPy array to a .npy file. In PerceptiLabs, you can then select that file for import when configuring a Data Component.

Alternatively, you can save multiple NumPy arrays to a .npz file using NumPy's numpy.savez method, and then select that file for import via PerceptiLabs' Data component.

PerceptiLabs supports NumPy files containing floating point data or other similar basic data types. Arbitrary Python objects are not supported.