Custom data

Updated by Ole Dallerup

For some companies, the standard integrations Dreamdata offers are not enough. There might be multiple reasons for this. For example, it might be because their product is a CRM, or because important customer information is stored in a custom system or in a system that Dreamdata does not yet integrate with. In these cases, it is necessary to build a custom integration with Dreamdata.

This page will describe how you build raw data files and share them with Dreamdata.

Before getting started, please contact your Dreamdata customer success manager. They can help you determine if this step is necessary and address any questions. Additionally, your customer success manager will connect you with the appropriate technical staff and provide you access to upload the files to the correct location.

File format

Files should be delivered in JSON newline delimited file (.ndjson). We highly recommend compressing the files using gzip.

Typical Implementation

  1. Build a schedule that triggers the data export job to run. 
  2. Run queries, export jobs, etc., to extract the data from where it’s currently stored. Currently, only full data dumps are supported.
  3. Transform the data and prepare the 4 files in the supported file formats.
  4. Compress the data using gzip.
  5. Upload the data. The data has to be uploaded to Google Cloud Storage. Here are code samples of how to upload data to Google Cloud Storage. Access will be shared via your customer success manager.
  6. Dreamdata will pick up the latest version of the data two times daily, at 04:00 am (UTC) and 04:00 pm (UTC).

Data files

Below are five tables, each describing a different file format. All fields within these tables are required, but you may leave some fields empty if you don't have any data or if it doesn't apply to your context.

It is not necessary to share a file for every table. For instance, you might only want to send the Stages or Subscriptions files, and sharing Events data may not be relevant to your case. Please consult your customer success manager if you are unsure which files to share or have other questions.

Companies

Fields

Type

Nullable?

Data Examples

Comment

id

string

FALSE

2394762270

Company unique id

created_date

timestamp

TRUE

2021-01-05T08:11:00.563Z

The timestamp of when the company was created in your system.

name

string

FALSE

Dreamdata

Name of the company

website

string

TRUE

dreamdata.io

The website of the company. It’s not a required field but highly recommended and Dreamdata works best when available.

additional_websites

array

TRUE

["dreamdataio.com", "dreamdata.cloud"]

Additional know websites associated with this company. Dreamdata will in some cases try to enrich this field using the primary website field.

country

string

TRUE

United States, France, Germany

The main country for the company. Dreamdata will automatically try to enrich this using the website field.

owner_name

string

TRUE

Sale Person

The name of the sales or customers success person who owns the account.

owner_email

string

TRUE

sales.person@dreamdata.io

The email of the sales or customers success person who owns the account.

industry

string

TRUE

Information Technology & Services, Computer Software, Financial Services, Real Estate, Telecommunications

In the companies industry, Dreamdata will automatically try to enrich this using the website field.

number_of_employees

string

TRUE

24,1200, 20000, 1 - 10, 10k - 50k

The number of employees within the company. If this is a number, Dreamdata will automatically bucket the data, if this is a bucket then that bucket will be used as-is. Dreamdata will automatically try to enrich this using the website field.

annual_revenue

string

TRUE

10500000, 200000000, 10M - 50M, 50M - 100M

The annual revenue of the company. If this is a number, Dreamdata will automatically bucket the data, if this is a bucket then that bucket will be used as is

type

string

TRUE

prospect, customer, competitor, partner

The type of the company

Contacts

Fields

Type

Nullable?

Data Examples

Comment

id

string

FALSE

39551

Contacts unique id

email

string

FALSE

friends@dreamdata.io

Primary email

additional_emails

array

TRUE

["support@dreamdata.io", "friends@dreamdataio.com"]

Any additional emails that might be associated with the contact

created_date

timestamp

TRUE

2021-01-26T12:23:00.563Z

The timestamp of when the contact was created in your system.

company_id

string

TRUE

2394762270

Company id pointing to the company the contact is associated with. If the contact is associated with more than one company, then duplicated contact records is expected

country

string

TRUE

United States, France, Germany

The contacts country

website

string

TRUE

dreamdata.io

The company website of the contact

name

string

TRUE

Ole Dallerup

The contacts name

first_name

string

TRUE

Ole

The contacts name

last_name

string

TRUE

Dallerup

The contacts name

role

string

TRUE

communications, customer_service, education, engineering, finance, health_professional, human_resources, information_technology, leadership, legal, marketing, operations, product, public_relations, real_estate, recruiting, sales

The contacts role

title

string

TRUE

VP of Marketing, CEO, Software Engineer, Finance Manager

The contacts title

seniority

string

TRUE

executive, director, manager

The contacts seniority

buying_role

string

TRUE

decision_maker, budget_holder, blocker

The contacts buying role

Stages

It depends on system, often this is called opportunities or deals. We call it stages, because they represent both very steps very early in the funnel and also later when it in fact is close becoming new business bookings.

Fields

Type

Nullable?

Data Examples

Comment

id

string

FALSE

4053091339

Unique id for the stage

company_id

string

TRUE*

2394762270

Company_id from the primary CRM, if not available see email.

*company_id or email is required

email

string

TRUE*

friends@dreamdata.io

The email of the user who did purchase. Dreamdata will use this email to find the company id from the primary crm. Its only used if company_id is not set.

*company_id or email is required

name

string

FALSE

Trial, MQL, SQL, NewBiz, MRR, ARR, Deal Created

The name of the stage. This value will be used to map the data into different revenue models inside Dreamdata.

amount

float64

FALSE

14999.99

The value on the stage

currency

string

FALSE

USD, EUR, GBP

The currency the amount represents. If you are sending mixed currencies Dreamdata will ensure to calculate all amount into your base currency.

created_date

timestamp

FALSE

2020-11-21T12:23:00.563Z

Timestamp of when the opportunity was created

close_date

timestamp

TRUE

2021-02-01T10:05:00.671Z

Close date of opportunity. If the value is not set, its expected that the stage has not closed yet.

stage

string

TRUE

won,lost

What stage the stage is currently in.

type

string

TRUE

newbusiness,renewal

The type of stage this is.

pipeline

string

TRUE

sales,bdr

The pipeline was in. When used you might have multiple entities of the same id with different pipeline_updated_at timestamps

pipeline_updated_at

timestamp

TRUE

2020-11-21T12:23:00.563Z

If pipeline is not set, then this field will be ignored. 

Subscriptions

If you are an pur subscription business this might be relevant for your to share subscriptions instead of Stages.

Fields

Type

Nullable?

Data Examples

Comment

id

string

FALSE

4053091339

Unique id for the subscription

company_id

string

TRUE*

2394762270

Company_id from the primary CRM, if not available see email.

*company_id or email is required

email

string

TRUE*

friends@dreamdata.io

The email of the user who did purchase. Dreamdata will use this email to find the company id from the primary crm. Its only used if company_id is not set.

*company_id or email is required

name

string

FALSE

Trial, MQL, SQL, NewBiz, MRR, ARR, Deal Created

The name of the subscription. This value will be used to map the data into different revenue models inside Dreamdata.

amount

float64

FALSE

14999.99

The value on the subscription

currency

string

FALSE

USD, EUR, GBP

The currency the amount represents. If you are sending mixed currencies Dreamdata will ensure to calculate all amount into your base currency.

start_date

timestamp

FALSE

2020-11-21T12:23:00.563Z

Start of billing period

end_date

timestamp

TRUE

2021-02-01T10:05:00.671Z

End of billing period

stage

string

TRUE

closedwon,closedlost

What stage the subscription is currently in.

type

string

TRUE

newbusiness,renewal

The type of subscription this is.

Events

Events are generic activities a user can do. See below for typical examples. 

Fields

Type

Nullable?

Data Examples

Comment

event_id

string

FALSE

Id of the event

event

string

FALSE

page_view, demo_request

Event name, as such this is free text. However if you use the typical event examples it will be easier to Dreamdata to guess what kind of an event this is.

timestamp

timestamp

FALSE

2020-11-21T12:23:00.563Z

Time of the event

anonymous_id

string

TRUE

Either anonymous_id, user_id or email has to be set

user_id

string

TRUE

Either anonymous_id, user_id or email has to be set

email

string

TRUE

friends@dreamdata.io

Either anonymous_id, user_id or email has to be set

url

string

TRUE

https://dreamdata.io/data-platform

referrer

string

TRUE

https://google.com/

utm_source

string

TRUE

google

utm_medium

string

TRUE

cpc

utm_campaign

string

TRUE

revenue_attribution

utm_term

string

TRUE

b2b attribution

ip

string

TRUE

127.0.0.1

The IP from where the event happened.

user_agent

string

TRUE

Typical event examples:

page_view

requested_demo

trial

sign_up

form_submission

subscribe

subscribe_newsletter

download_content

download_ebook

login

email_sent

email_opened

email_clicked

email_replied

inbound_phone_call

outbound_phone_call

phone_call

meeting

register_conference

attended_conference

register_event

attended_event

register_webinar

attended_webinar

watched_video

conversation

lead_referral

 

How did we do?