Custom data
For some companies, the standard integrations Dreamdata offers are not enough. There might be multiple reasons for this. For example, it might be because their product is a CRM, or because important customer information is stored in a custom system or in a system that Dreamdata does not yet integrate with. In these cases, it is necessary to build a custom integration with Dreamdata.
This page will describe how you build raw data files and share them with Dreamdata.
Before getting started, please contact your Dreamdata customer success manager. They can help you determine if this step is necessary and address any questions. Additionally, your customer success manager will connect you with the appropriate technical staff and provide you access to upload the files to the correct location.
File format
Files should be delivered in JSON newline delimited file (.ndjson). We highly recommend compressing the files using gzip.
Typical Implementation
- Build a schedule that triggers the data export job to run.
- Run queries, export jobs, etc., to extract the data from where it’s currently stored. Currently, only full data dumps are supported.
- Transform the data and prepare the 4 files in the supported file formats.
- Compress the data using gzip.
- Upload the data. The data has to be uploaded to Google Cloud Storage. Here are code samples of how to upload data to Google Cloud Storage. Access will be shared via your customer success manager.
- Dreamdata will pick up the latest version of the data two times daily, at 04:00 am (UTC) and 04:00 pm (UTC).
Data files
Below are five tables, each describing a different file format. All fields within these tables are required, but you may leave some fields empty if you don't have any data or if it doesn't apply to your context.
It is not necessary to share a file for every table. For instance, you might only want to send the Stages or Subscriptions files, and sharing Events data may not be relevant to your case. Please consult your customer success manager if you are unsure which files to share or have other questions.
Companies
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 2394762270 | Company unique id |
created_date | timestamp | TRUE | 2021-01-05T08:11:00.563Z | The timestamp of when the company was created in your system. |
name | string | FALSE | Dreamdata | Name of the company |
website | string | TRUE | The website of the company. It’s not a required field but highly recommended and Dreamdata works best when available. | |
additional_websites | array | TRUE | ["dreamdataio.com", "dreamdata.cloud"] | Additional know websites associated with this company. Dreamdata will in some cases try to enrich this field using the primary website field. |
country | string | TRUE | United States, France, Germany | The main country for the company. Dreamdata will automatically try to enrich this using the website field. |
owner_name | string | TRUE | Sale Person | The name of the sales or customers success person who owns the account. |
owner_email | string | TRUE | sales.person@dreamdata.io | The email of the sales or customers success person who owns the account. |
industry | string | TRUE | Information Technology & Services, Computer Software, Financial Services, Real Estate, Telecommunications | In the companies industry, Dreamdata will automatically try to enrich this using the website field. |
number_of_employees | string | TRUE | 24,1200, 20000, 1 - 10, 10k - 50k | The number of employees within the company. If this is a number, Dreamdata will automatically bucket the data, if this is a bucket then that bucket will be used as-is. Dreamdata will automatically try to enrich this using the website field. |
annual_revenue | string | TRUE | 10500000, 200000000, 10M - 50M, 50M - 100M | The annual revenue of the company. If this is a number, Dreamdata will automatically bucket the data, if this is a bucket then that bucket will be used as is |
type | string | TRUE | prospect, customer, competitor, partner | The type of the company |
Contacts
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 39551 | Contacts unique id |
string | FALSE | friends@dreamdata.io | Primary email | |
additional_emails | array | TRUE | ["support@dreamdata.io", "friends@dreamdataio.com"] | Any additional emails that might be associated with the contact |
created_date | timestamp | TRUE | 2021-01-26T12:23:00.563Z | The timestamp of when the contact was created in your system. |
company_id | string | TRUE | 2394762270 | Company id pointing to the company the contact is associated with. If the contact is associated with more than one company, then duplicated contact records is expected |
country | string | TRUE | United States, France, Germany | The contacts country |
website | string | TRUE | The company website of the contact | |
name | string | TRUE | Ole Dallerup | The contacts name |
first_name | string | TRUE | Ole | The contacts name |
last_name | string | TRUE | Dallerup | The contacts name |
role | string | TRUE | communications, customer_service, education, engineering, finance, health_professional, human_resources, information_technology, leadership, legal, marketing, operations, product, public_relations, real_estate, recruiting, sales | The contacts role |
title | string | TRUE | VP of Marketing, CEO, Software Engineer, Finance Manager | The contacts title |
seniority | string | TRUE | executive, director, manager | The contacts seniority |
buying_role | string | TRUE | decision_maker, budget_holder, blocker | The contacts buying role |
Stages
It depends on system, often this is called opportunities or deals. We call it stages, because they represent both very steps very early in the funnel and also later when it in fact is close becoming new business bookings.
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 4053091339 | Unique id for the stage |
company_id | string | TRUE* | 2394762270 | Company_id from the primary CRM, if not available see email. *company_id or email is required |
string | TRUE* | friends@dreamdata.io | The email of the user who did purchase. Dreamdata will use this email to find the company id from the primary crm. Its only used if company_id is not set. *company_id or email is required | |
name | string | FALSE | Trial, MQL, SQL, NewBiz, MRR, ARR, Deal Created | The name of the stage. This value will be used to map the data into different revenue models inside Dreamdata. |
amount | float64 | FALSE | 14999.99 | The value on the stage |
currency | string | FALSE | USD, EUR, GBP | The currency the amount represents. If you are sending mixed currencies Dreamdata will ensure to calculate all amount into your base currency. |
created_date | timestamp | FALSE | 2020-11-21T12:23:00.563Z | Timestamp of when the opportunity was created |
close_date | timestamp | TRUE | 2021-02-01T10:05:00.671Z | Close date of opportunity. If the value is not set, its expected that the stage has not closed yet. |
stage | string | TRUE | won,lost | What stage the stage is currently in. |
type | string | TRUE | newbusiness,renewal | The type of stage this is. |
pipeline | string | TRUE | sales,bdr | The pipeline was in. When used you might have multiple entities of the same id with different pipeline_updated_at timestamps |
pipeline_updated_at | timestamp | TRUE | 2020-11-21T12:23:00.563Z | If pipeline is not set, then this field will be ignored. |
Subscriptions
If you are an pur subscription business this might be relevant for your to share subscriptions instead of Stages.
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 4053091339 | Unique id for the subscription |
company_id | string | TRUE* | 2394762270 | Company_id from the primary CRM, if not available see email. *company_id or email is required |
string | TRUE* | friends@dreamdata.io | The email of the user who did purchase. Dreamdata will use this email to find the company id from the primary crm. Its only used if company_id is not set. *company_id or email is required | |
name | string | FALSE | Trial, MQL, SQL, NewBiz, MRR, ARR, Deal Created | The name of the subscription. This value will be used to map the data into different revenue models inside Dreamdata. |
amount | float64 | FALSE | 14999.99 | The value on the subscription |
currency | string | FALSE | USD, EUR, GBP | The currency the amount represents. If you are sending mixed currencies Dreamdata will ensure to calculate all amount into your base currency. |
start_date | timestamp | FALSE | 2020-11-21T12:23:00.563Z | Start of billing period |
end_date | timestamp | TRUE | 2021-02-01T10:05:00.671Z | End of billing period |
stage | string | TRUE | closedwon,closedlost | What stage the subscription is currently in. |
type | string | TRUE | newbusiness,renewal | The type of subscription this is. |
Events
Events are generic activities a user can do. See below for typical examples.
Fields | Type | Nullable? | Data Examples | Comment |
event_id | string | FALSE | Id of the event | |
event | string | FALSE | page_view, demo_request | Event name, as such this is free text. However if you use the typical event examples it will be easier to Dreamdata to guess what kind of an event this is. |
timestamp | timestamp | FALSE | 2020-11-21T12:23:00.563Z | Time of the event |
anonymous_id | string | TRUE | Either anonymous_id, user_id or email has to be set | |
user_id | string | TRUE | Either anonymous_id, user_id or email has to be set | |
string | TRUE | friends@dreamdata.io | Either anonymous_id, user_id or email has to be set | |
url | string | TRUE | https://dreamdata.io/data-platform | |
referrer | string | TRUE | https://google.com/ | |
utm_source | string | TRUE | ||
utm_medium | string | TRUE | cpc | |
utm_campaign | string | TRUE | revenue_attribution | |
utm_term | string | TRUE | b2b attribution | |
ip | string | TRUE | 127.0.0.1 | The IP from where the event happened. |
user_agent | string | TRUE |
Typical event examples:
page_view |
requested_demo |
trial |
sign_up |
form_submission |
subscribe |
subscribe_newsletter |
download_content |
download_ebook |
login |
email_sent |
email_opened |
email_clicked |
email_replied |
inbound_phone_call |
outbound_phone_call |
phone_call |
meeting |
register_conference |
attended_conference |
register_event |
attended_event |
register_webinar |
attended_webinar |
watched_video |
conversation |
lead_referral |
