Custom Integration
For some companies the standard integrations Dreamdata offers are not enough. There might be multiple reasons for this. For example, it might be because their product is an CRM, or it might be because important customers information is stored in a custom system or in a system that Dreamdata does not yet integrate with. In these cases, it is necessary to build a custom integration with Dreamdata.
This page will describe how you build raw data files and share them with Dreamdata.
Before getting started you should reach out with your Dreamdata customer success manager. So he/she can help you deciding if this is required and answer any questions. Your customer success manager will also help ensuring to connect you with technical staff and access for you to upload the files the right place.
File format
Files should be delivered in JSON newline delimited file (.ndjson). We highly recommend compressing the files using gzip.
Typical Implementation
- Build a schedule that triggers the data export job to run.
- Run queries, export jobs, etc. to extract the data from where it’s currently stored. Currently only full data dumps is supported.
- Transform the data and prepare the 4 different files in the supported file formats
- Compresse the data using gzip.
- Upload the data. The data has to be uploaded to Google Cloud Storage. Here is code samples of how to upload data to Google Cloud Storage. Access will be shared via your customer success manager.
- Dreamdata will pick up the latest version the data 4 times per day, at 3:30am, 09:45 am, 03:45 pm and 09:45 pm (UTC). We typically recommend you update the data daily for pickup at 3:30am (UTC).
Data files
Here below is 5 tables each describing the file format. All fields are required, however many of them you don't have to set any value if you either don't have any data or it does not make sense in your context.
It's also not required to share share a file for each table. First you only want to send either Stages or Subscriptions. However it might not be relevant for you to share events data example. Have a talk with your customer success manager if you are in doubt.
Companies
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 2394762270 | Company unique id |
created_date | timestamp | TRUE | 2021-01-05T08:11:00.563Z | The timestamp of when the company was created in your system. |
name | string | FALSE | Dreamdata | Name of the company |
website | string | TRUE | The website of the company. It’s not a required field but highly recommended and Dreamdata works best when available. | |
additional_websites | array | TRUE | ["dreamdataio.com", "dreamdata.cloud"] | Additional know websites associated with this company. Dreamdata will in some cases try to enrich this field using the primary website field. |
country | string | TRUE | United States, France, Germany | The main country for the company. Dreamdata will automatically try to enrich this using the website field. |
owner_name | string | TRUE | Sale Person | The name of the sales or customers success person who owns the account. |
owner_email | string | TRUE | sales.person@dreamdata.io | The email of the sales or customers success person who owns the account. |
industry | string | TRUE | Information Technology & Services, Computer Software, Financial Services, Real Estate, Telecommunications | In the companies industry, Dreamdata will automatically try to enrich this using the website field. |
number_of_employees | string | TRUE | 24,1200, 20000, 1 - 10, 10k - 50k | The number of employees within the company. If this is a number, Dreamdata will automatically bucket the data, if this is a bucket then that bucket will be used as-is. Dreamdata will automatically try to enrich this using the website field. |
annual_revenue | string | TRUE | 10500000, 200000000, 10M - 50M, 50M - 100M | The annual revenue of the company. If this is a number, Dreamdata will automatically bucket the data, if this is a bucket then that bucket will be used as is |
type | string | TRUE | prospect, customer, competitor, partner | The type of the company |
Contacts
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 39551 | Contacts unique id |
string | FALSE | friends@dreamdata.io | Primary email | |
additional_emails | array | TRUE | ["support@dreamdata.io", "friends@dreamdataio.com"] | Any additional emails that might be associated with the contact |
created_date | timestamp | TRUE | 2021-01-26T12:23:00.563Z | The timestamp of when the contact was created in your system. |
company_id | string | TRUE | 2394762270 | Company id pointing to the company the contact is associated with. If the contact is associated with more than one company, then duplicated contact records is expected |
country | string | TRUE | United States, France, Germany | The contacts country |
website | string | TRUE | The company website of the contact | |
name | string | TRUE | Ole Dallerup | The contacts name |
first_name | string | TRUE | Ole | The contacts name |
last_name | string | TRUE | Dallerup | The contacts name |
role | string | TRUE | communications, customer_service, education, engineering, finance, health_professional, human_resources, information_technology, leadership, legal, marketing, operations, product, public_relations, real_estate, recruiting, sales | The contacts role |
title | string | TRUE | VP of Marketing, CEO, Software Engineer, Finance Manager | The contacts title |
seniority | string | TRUE | executive, director, manager | The contacts seniority |
buying_role | string | TRUE | decision_maker, budget_holder, blocker | The contacts buying role |
Stages
It depends on system, often this is called opportunities or deals. We call it stages, because they represent both very steps very early in the funnel and also later when it in fact is close becoming new business bookings.
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 4053091339 | Unique id for the stage |
company_id | string | TRUE* | 2394762270 | Company_id from the primary CRM, if not available see email. *company_id or email is required |
string | TRUE* | friends@dreamdata.io | The email of the user who did purchase. Dreamdata will use this email to find the company id from the primary crm. Its only used if company_id is not set. *company_id or email is required | |
name | string | FALSE | Trial, MQL, SQL, NewBiz, MRR, ARR, Deal Created | The name of the stage. This value will be used to map the data into different revenue models inside Dreamdata. |
amount | float64 | FALSE | 14999.99 | The value on the stage |
currency | string | FALSE | USD, EUR, GBP | The currency the amount represents. If you are sending mixed currencies Dreamdata will ensure to calculate all amount into your base currency. |
created_date | timestamp | FALSE | 2020-11-21T12:23:00.563Z | Timestamp of when the opportunity was created |
close_date | timestamp | TRUE | 2021-02-01T10:05:00.671Z | Close date of opportunity. If the value is not set, its expected that the stage has not closed yet. |
stage | string | TRUE | won,lost | What stage the stage is currently in. |
type | string | TRUE | newbusiness,renewal | The type of stage this is. |
pipeline | string | TRUE | sales,bdr | The pipeline was in. When used you might have multiple entities of the same id with different pipeline_updated_at timestamps |
pipeline_updated_at | timestamp | TRUE | 2020-11-21T12:23:00.563Z | If pipeline is not set, then this field will be ignored. |
Subscriptions
If you are an pur subscription business this might be relevant for your to share subscriptions instead of Stages.
Fields | Type | Nullable? | Data Examples | Comment |
id | string | FALSE | 4053091339 | Unique id for the subscription |
company_id | string | TRUE* | 2394762270 | Company_id from the primary CRM, if not available see email. *company_id or email is required |
string | TRUE* | friends@dreamdata.io | The email of the user who did purchase. Dreamdata will use this email to find the company id from the primary crm. Its only used if company_id is not set. *company_id or email is required | |
name | string | FALSE | Trial, MQL, SQL, NewBiz, MRR, ARR, Deal Created | The name of the subscription. This value will be used to map the data into different revenue models inside Dreamdata. |
amount | float64 | FALSE | 14999.99 | The value on the subscription |
currency | string | FALSE | USD, EUR, GBP | The currency the amount represents. If you are sending mixed currencies Dreamdata will ensure to calculate all amount into your base currency. |
start_date | timestamp | FALSE | 2020-11-21T12:23:00.563Z | Start of billing period |
end_date | timestamp | TRUE | 2021-02-01T10:05:00.671Z | End of billing period |
stage | string | TRUE | closedwon,closedlost | What stage the subscription is currently in. |
type | string | TRUE | newbusiness,renewal | The type of subscription this is. |
Events
Events are generic activities a user can do. See below for typical examples.
Fields | Type | Nullable? | Data Examples | Comment |
event_id | string | FALSE | Id of the event | |
event | string | FALSE | page_view, demo_request | Event name, as such this is free text. However if you use the typical event examples it will be easier to Dreamdata to guess what kind of an event this is. |
timestamp | timestamp | FALSE | 2020-11-21T12:23:00.563Z | Time of the event |
anonymous_id | string | TRUE | Either anonymous_id, user_id or email has to be set | |
user_id | string | TRUE | Either anonymous_id, user_id or email has to be set | |
string | TRUE | friends@dreamdata.io | Either anonymous_id, user_id or email has to be set | |
url | string | TRUE | https://dreamdata.io/data-platform | |
referrer | string | TRUE | https://google.com/ | |
utm_source | string | TRUE | ||
utm_medium | string | TRUE | cpc | |
utm_campaign | string | TRUE | revenue_attribution | |
utm_term | string | TRUE | b2b attribution | |
ip | string | TRUE | 127.0.0.1 | The IP from where the event happened. |
user_agent | string | TRUE |
Typical event examples:
page_view |
requested_demo |
trial |
sign_up |
form_submission |
subscribe |
subscribe_newsletter |
download_content |
download_ebook |
login |
email_sent |
email_opened |
email_clicked |
email_replied |
inbound_phone_call |
outbound_phone_call |
phone_call |
meeting |
register_conference |
attended_conference |
register_event |
attended_event |
register_webinar |
attended_webinar |
watched_video |
conversation |
lead_referral |
