Array / Map operations¶
As explained in the Cleaning operation, there are a variety of different transformation steps available for the user in order to enhance the experience of cleaning your datasets. One of these types of transformations includes as well cleaning operations on Array and Map type columns. These complex types of data can be found in some cases through JSON type APIs and can be challenging to extract and structure a good schema from it.
Hence, papAI offers you multiple operations specifically made on these data types to speed up the process efficiently.
To access to these specific operations, you simply follow the same step as to apply a cleaning step : select the desired dataset on your flow and click on the cleaning icon on the left sidebar to access to the cleaning module.
The only difference than regular operations is the choose from the list of operations the Array or the Map type cleaning operations. These operations include extracting keys and values, filter some values, sorting, transform array or map into a string, ensemble operation on multiple arrays or maps...
The list of available operations for Arrays is the following :
Here is the same list but for Maps :
You just apply what step is necessary and adjust the settings to what suits you best and when have done all the necessary changes, you click on the green Submit and Run button to apply the changes on a new output dataset.
