How to resolve the error - Object of type bytes is not JSON serializable?
I am attempting to move ContentVersion records from one Salesforce org to another using the simple-salesforce python library.
I am querying the VersionData and returning it in r.content here:
import json import requests import base64 import chardet headers = { "Authorization": "OAuth " + source_sf.session_id, "Content-Type": "application/octet-stream" } for index, row in content_doc_source_data_df.iterrows(): r = requests.request('get', source_sf.base_url+'sobjects/ContentVersion/'+row['Id']+'/VersionData', headers=headers) content_doc_source_data_df.at[index, 'File Body'] = r.content break content_doc_source_data_df.drop('Id', axis=1, inplace=True) content_doc_source_data_df.drop('ContentDocumentId', axis=1, inplace=True) content_doc_source_data_df.drop('ContentLocation', axis=1, inplace=True) content_doc_source_data_df.drop('ContentUrl', axis=1, inplace=True) content_doc_source_data_df.drop('IsLatest', axis=1, inplace=True) content_doc_source_data_df.drop('VersionData', axis=1, inplace=True) content_doc_source_data_df = content_doc_source_data_df.rename(columns={'File Body': 'VersionData'}) display(content_doc_source_data_df)Which I can tell is returning the bites data, which you can see here:
However, whenever I then attempt to resend it to the new org using the API I get the following error: "TypeError: Object of type bytes is not JSON serializable"
Here's the code for sending:
#INSERT FILE import ast content_doc_source_data = content_doc_source_data_df.to_dict('records') results = source_sf.bulk.ContentVersion.insert(content_doc_source_data, batch_size=10000,use_serial=True) display(results)You should be following the steps found in Inserting a Document with Blob Data to resolve the error - Object of type bytes is not JSON serializable. That is, your payload must be a multipart/form-data content, with one element called entity_document with the appropriate JSON, and a second element that has the name VersionData with binary data.
You can't upload files using the Bulk API. Even if you could, note that the limits are pretty strict:
A batch can contain a maximum of 10,000,000 characters for all the data in a batch.
A field can contain a maximum of 32,000 characters.Are your PDF files smaller than about 24KB? If not, they won't fit in the field limit. To be honest, I don't think you can use simple-salesforce here (I think that's what you're using?). You'll want to just manually construct a multipart/form-data payload and use the appropriate endpoint (POST /services/data/v55.0/ContentVersion).