property source is s3 Schema #
undefined#/properties/postProcess/items/allOf/21/properties/params/properties/dataSource/oneOf/1/allOf/0
Abstract | Extensible | Status | Identifiable | Custom Properties | Additional Properties | Access Restrictions | Defined In |
---|---|---|---|---|---|---|---|
Can be instantiated | No | Unknown status | No | Forbidden | Allowed | none | specSchema.schema.json* |
0 Type #
unknown (property source is s3)
property source is s3 Properties #
awsAccess #
AWS Access Credentials
awsAccess
timePrefix #
timePrefix TODO
timePrefix
- is optional
- type: any of the folllowing:
string
orarray
(timePrefix) - cannot be null
- defined in: timePrefix
bucket #
Top level S3 bucket under which to find data, metadata, or namespace folders
bucket
- is optional
- type:
string
- cannot be null
- Unstructured type
namespace #
Namespace separation of data files. Folder path under top level S3 bucket in which to find related data files
namespace
- is optional
- type:
string
- cannot be null
- Unstructured type
s3ListObjectsAll #
s3ListObjectsAll TODO
s3ListObjectsAll
- is optional
- type:
boolean
- cannot be null
- Unstructured type
objectTimeRange #
objectTimeRange TODO
objectTimeRange
- is optional
- type:
string
- cannot be null
- Unstructured type
s3Sql #
S3 SQL statement to select rows of data to be returned from the files in the dataset
s3Sql
- is optional
- type:
string
- cannot be null
- Unstructured type
suffixFilter #
Only process objects whose key ends with the suffixFilter
suffixFilter
- is optional
- type:
string
- cannot be null
- Unstructured type
maxObjects #
maxObjects TODO
maxObjects
- is optional
- type:
number
- cannot be null
- Unstructured type
maxObjectsDirection #
Enumeration of strings with constraints below
maxObjectsDirection
- is optional
- type: unknown (maxObjectsDirection)
- cannot be null
- defined in: maxObjectsDirection
maxObjectsDirection Constraints #
enum: the value of this property must be equal to one of the following values:
Value | Explanation |
---|---|
"left" |
|
"right" |
style #
Enumeration of strings with constraints below
style
style Constraints #
enum: the value of this property must be equal to one of the following values:
Value | Explanation |
---|---|
"conventional" |
Line delimitted JSON, gzipped. Filename ending: .jsonl.gz |
"kinesis-firehose" |
Line delimitted JSON, uncompressed text. Filename ending: .gz |
extractTime #
JSONata expression to extract time from S3 object key, used by objectTimeRange filtering
extractTime
- is optional
- type:
string
- cannot be null
- Unstructured type