Gigya Job Openings

Azure SAS Dataflow

Skip to end of metadata
Go to start of metadata

The dataflow that loads data from Gigya to Azure using a shared access signature (SAS) includes field renaming, field removal, formatting the file in an Azure-compatible format, and writing the data to Azure. The various dataflow steps are described in greater detail in the Component Repository

Note that IdentitySync jobs are scheduled in UTC time. Therefore, the platform participating in the flow should be set to the UTC timezone to ensure that file requests are handled properly.

Following is a code sample of a full dataflow for passing data from Gigya to Azure: 

 {
	"name" : "Azure - SAS",
	"steps" : [{
			"id" : "account",
			"type" : "datasource.read.gigya.account",
			"params" : {
				"select" : "UID,profile"
			},
			"next" : ["rename"]
		}, {
			"id" : "rename",
			"type" : "field.rename",
			"params" : {
				"fields" : [{
						"sourceField" : "profile.relationshipStatus",
						"targetField" : "relationshipStatus"
					}, {
						"sourceField" : "profile.gender",
						"targetField" : "gender"
					}, {
						"sourceField" : "profile.birthDay",
						"targetField" : "birthDay"
					}, {
						"sourceField" : "profile.birthMonth",
						"targetField" : "birthMonth"
					}, {
						"sourceField" : "profile.birthYear",
						"targetField" : "birthYear"
					}, {
						"sourceField" : "profile.age",
						"targetField" : "age"
					}, {
						"sourceField" : "profile.education",
						"targetField" : "education"
					}, {
						"sourceField" : "profile.educationLevel",
						"targetField" : "educationLevel"
					}, {
						"sourceField" : "profile.followersCount",
						"targetField" : "followersCount"
					}, {
						"sourceField" : "profile.followingCount",
						"targetField" : "followingCount"
					}, {
						"sourceField" : "profile.industry",
						"targetField" : "industry"
					}
				]
			},
			"next" : ["remove"]
		}, {
			"id" : "remove",
			"type" : "field.remove",
			"params" : {
				"fields" : ["profile"]
			},
			"next" : ["dsv"]
		}, {
			"id" : "dsv",
			"type" : "file.format.dsv",
			"params" : {
				"fileName" : "testAzure${now}.csv",
				"columnSeparator" : ","
			},
			"next" : ["azure"]
		}, {
			"id" : "azure",
			"type" : "datasource.write.azure.sas",
			"params" : {
				"clientId" : "...",
				"clientSecret" : "...",
				"loginUrl" : "https://login.microsoftonline.com/company.onmicrosoft.com",
				"authenticationUrl" : "http://companysasmanager.azurewebsites.net",
				"blobUri" : "/api/sasstorage/create/"
			}
		}
	]
}

  • No labels