Gigya Job Openings

Page History

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand
titledatasource.read.gigya.account

Extract Account data from Gigya.

  • By default, the component extracts accounts that have been updated (modified) since the last time the dataflow was executed successfully. This behavior may be changed using the deltaField parameter.
  • Additional conditions may be added using the where parameter.

This component calls the accounts.search method.

RequiredParameter NameTypeDefaultDescription
Yes
selectstring 

Comma-separated list of fields to extract. For example, enter "UID, profile.firstName" to extract the 'UID' and 'profile.firstName' fields from the Account object.

No
fromstringaccountsThe name of the data source.
No
wherestring 

An SQL-like WHERE clause to filter Account objects.

You can add an indication of time to the query in a highly flexible manner, to retrieve only records that were updated in a specific time period. For example, to retrieve users that started but did not complete the registration process in the last hour, the following query applies (see also a full dataflow below):

Code Block
"where" : "isRegistered = false AND createdTimestamp < ${now-1h}"

The supported time units are:

  • y (years)
  • M (months)
  • w (weeks)
  • d (days)
  • h (hours)
  • m (minutes)
  • s (seconds)
Note

The time indicator can be used only when the schedule is set to retrieve all records, i.e. fullExtract is set to 'true'.

No
deltaFieldstringlastUpdatedTimestamp

The timestamp field to filter accounts by. The component will extract accounts where this timestamp is newer than the timestamp of the last successful run of the dataflow.

Accepted values:

  • lastUpdatedTimestamp

  • lastLoginTimestamp

  • oldestDataUpdatedTimestamp

  • createdTimestamp

  • verifiedTimestamp

  • registeredTimestamp

No
consentarray 

For customers who are using Customer Consent, an array of consent statements and their statuses, by which to filter the queried users. For example, you can retrieve only users who have a valid consent statement assigned to their account.

Both parameters are required. The parameters for each statement are:

  • id (string): the identifier of the consent statement.
  • status (string): the status of this statement in the user's account. Possible values are:
    • valid: the user has given consent to this statement and that consent is still valid.
    • expired: the user has given consent to this statement in the past, but due to a version change the consent has expired.
    • notGranted: the user has never given consent to this statement, or has withdrawn that consent.
No
maxConcurrencyinteger1

The maximum number of threads to allocate when executing a search in parallel. Accepts values between 1 and 16 (default is 1).

Note

The default value of "1" is the best practice for most scenarios. Unless performance is a critical consideration for the job you are running, this parameter should not be changed.

Varies
apiKeystring 

When copying accounts from one site to another, the API key of the site from which to read the data.

Note

All three parameters are required when copying site accounts: apiKey, userKey and secret.

Varies
userKeystring 

When copying accounts from one site to another, the user or application key used for authentication.

Note

All three parameters are required when copying site accounts: apiKey, userKey and secret.


Varies
secretstring 

When copying accounts from one site to another, the user or application secret used for authentication.

Note

All three parameters are required when copying site accounts: apiKey, userKey and secret.


No
keepFieldNamesWithDotAsIsBooleantrueWhen set to false, field names that contain dots are read as nested properties. When set to true, field names with dots are treated as a single field name.

Following is a sample dataflow that demonstrates using datasource.read.gigya.account to load subscription status from Gigya into Mailchimp:

Code Block
languagejs
linenumberstrue
{
    "id" : "",
    "name" : "Subscription management",
    "description" : "account > rename > mailchimp",
    "steps" : [{
            "id" : "account",
            "type" : "datasource.read.gigya.account",
            "params" : {
                "select" : "subscriptions.mySub.email.isSubscribed",
                "from" : "emailAccounts"
            },
            "next" : ["rename"]
        }, {
            "id" : "rename",
            "type" : "field.rename",
            "params" : {
                "fields" : [{
                        "sourceField" : "subscriptions.mySub.email.isSubscribed",
                        "targetField" : "newsletterField"
                    }
                ]
            },
            "next" : ["mailchimp"]
        }, {
            "id" : "mailchimp",
            "type" : "datasource.write.mailchimp",
            "params" : {
                "apiUrl" : "https://<dc>.api.mailchimp.com/3.0/",
                "apiKey" : "",
                "listId" : "",
                "newsletterField" : "newsletterField"
            }
        }
    ]
}

 

The following dataflow demonstrates using a flexible time notation in the WHERE clause

Code Block
languagejs
linenumberstrue
{
    "name" : "Fetch unregistered accounts",
    "description" : "account > delete",
    "steps" : [{
            "id" : "account",
            "type" : "datasource.read.gigya.account",
            "params" : {
                "select" : "UID,isRegistered,createdTimestamp"
                "where" : "isRegistered = false AND createdTimestamp < ${now-1h}"
            },
            "next" : ["remove"]
        }, {
            "id" : "remove",
            "type" : "field.remove",
            "params" : {
                "fields" : ["isRegistered","createdTimestamp"]
            },
            "next" : ["dsv"]
        }, {
            "id" : "dsv",
            "type" : "file.format.dsv",
            "params" : {
                "fileName" : "GIGYA_TO_SFTP_${now}.csv",
                "columnSeparator" : ",",
                "quoteFields" : true
            },
            "next" : ["gzip"]
        }, {
            "id" : "gzip",
            "type" : "file.compress.gzip",
            "next" : ["sftp"]
        }, {
            "id" : "sftp",
            "type" : "datasource.write.sftp",
            "params" : {
                "host" : "idx-etl",
                "username" : "idx",
                "password" : "",
                "remotePath" : "test"
            }
        }
    ]
}

 

The following dataflow demonstrates retrieving accounts based on 3 different statuses of consent statements:

Code Block
languagejs
linenumberstrue
{
 "name": "Export from Gigya to SFTP",
 "description": "account > rename > dsv > sftp",
 "steps": [
  {
   "id": "account",
   "type": "datasource.read.gigya.account",
   "params": {
    "select": "profile.email,profile.lastName,profile.firstName,profile.gender",
    "batchSize": 300,
    "from": "accounts",
    "deltaField": "lastUpdatedTimestamp",
    "maxConcurrency": 1,
    "consent":[
      {
        "id":"terms.tos",
        "status":"valid"
      },
      {
        "id":"terms.allowAds",
        "status":"expired"
      },
      {         
        "id":"terms.optionalConsent",
         "status":"notGranted"       
      }
    ]
   },
   "next": [
    "rename"
   ]
  },
  {
   "id": "rename",
   "type": "field.rename",
   "params": {
    "fields": [
     {
      "sourceField": "profile.email",
      "targetField": "MAIL"
     },
     {
      "sourceField": "profile.lastName",
      "targetField": "NAME"
     },
     {
      "sourceField": "profile.firstName",
      "targetField": "FIRSTNAME"
     },
     {
      "sourceField": "profile.gender",
      "targetField": "GENDER"
     }
    ]
   },
   "next": [  
    "dsv"
   ]
  },
  {
   "id": "dsv",
   "type": "file.format.dsv",
   "params": {
    "fileName": "GIGYA_TO_SFTP_${now}.csv",
    "columnSeparator": ",",
    "quoteFields": true,
    "writeHeader": true,
    "lineEnd": "\n",
    "createEmptyFile": false
   },
   "next": [
    "sftp"
   ]
  },
  {
   "id": "sftp",
   "type": "datasource.write.sftp",
   "params": {
    "host": "...",
    "username": "...",
    "password": "...",
    "remotePath": "GigyaDaily",
    "port": 22,
    }
  }
 ]
}

...