Data sources ingestion API
Leverage this API to programmatically ingest data into an existing Data Source
REST API
Data source asynchronous ingestion call
Follow this 3-step process to upload a dataset into an existing Data source.
Step 1: Generate an ingestion URL
Step 2: Upload file using the URL from Step 1
Step 3: Initiate ingestion
Step 1: Generate an ingestion URL
Step 2: Upload file using the URL from Step 1
Step 3: Initiate ingestion
Step 1: Generate an ingestion URL
Inputs
Input query parameters
key
required
input filename
type
required
datasource
Example of Rest API Call
curl https://{{ tenant }}.lightenup.ai/admin/s3_uploadlink \
\
--request POST \
--header "Authorization: Bearer {access_token}" \
--header "Content-Type: application/json" \
--data @- << EOF
{
"key": "{filename}",
"type": "datasource"
}
EOF
Responses
Response Code
200
Example of 200 response code
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
body: {"url: string"}
}Response Code
400
Example of 400 error
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
400,
{"message":"Missing parameters"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Missing parameters"}
}
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
400,
{"message":"Missing parameters"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Missing parameters"}
}
Response Code
500
Example of 500 error
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
500,
{"message":"Internal server: error processing intake step"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Internal server: error
processing intake step"}
}
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
500,
{"message":"Internal server: error processing intake step"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Internal server: error
processing intake step"}
}
Step 2: Upload file using the URL from Step 1
Inputs
Input query parameters
url
required
url from Step 1
file
required
file (maximum 6MB)
Example of Rest API Call
curl {url} \
\
--request PUT \
--header "Content-Type {type}" \
--data @- << EOF
{
"url": "https://profile-console-data-lighte...."
file
}
Responses
Response Code
200
Example of 200 response
// no content in bodyResponse Code
400
Example of 400 error
// no content in bodyResponse Code
500
Example of 500 error
// no content in body
Step 3: Initiate ingestion
Inputs
Input query parameters
data_source
required
datasource name
type
required
datasource type
uploadurl
required
url from Step 1
Example of Rest API Call
curl https://{{ tenant }}.lightenup.ai/admin/data/ingest/text_async \
\
--request POST \
--header "Authorization: Bearer {access_token}" \
--header "Content-Type: application/json" \
--data @- << EOF
{
"data_source": "catalog",
"type": "csv",
"uploadurl": "https://profile-console-data-lightenup.."
}
Responses
Response Code
200
Example of 200 response
{
"message": "Data ingestion in progress",
"ingestion_id": "f3170179-755e-43f8-b8ea-a544256e404c"
}Response Code
400
Example of 400 error
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
400,
{"message":"Missing parameters"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Missing parameters"}
}
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
400,
{"message":"Missing parameters"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Missing parameters"}
}
Response Code
500
Example of 500 error
{
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
500,
{"message":"Internal server: error processing intake step"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Internal server: error
processing intake step"}
}
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Methods': allowMethods,
'Access-Control-Allow-Origin': GLOBALS.CORS_ORIGIN,
'Access-Control-Allow-Credentials': true
},
500,
{"message":"Internal server: error processing intake step"},
Error: "CustomError",
Cause: {"errorType": "CustomError", "errorMessage": "Internal server: error
processing intake step"}
}