Glue

Create Glue

We will create Glue Jobs, Glue Crawler, Glue Database

Create Glue Database

1. Access the Glue console.:

   • Access the Glue console.
   • Select Database.
   • Select Add Database.

   

   • Enter your database name
   • Enter your staging bucket URI
   • Select Create Database.

   

Create Glue Crawler

1. Set crawler properties, data source:

   • Access the Glue console.
   • Select Crawler.
   • Select Create crawler.

   

   • Enter your crawler name
   • Select Next.
   • Select Add a data source.
   • Select your destination folder in staging bucket.
   • Select Add an S3 data source.

   

2. Set security:

   • Select your IAM glue role.
   • Select Next.

   

3. Set output:

   • Select your staging database.
   • Select Next.

   

   

Create Glue Jobs

1. Create Glue Jobs.:

   • Access the Glue console.
   • Select ETL Jobs.
   • Select Visual ETL.

   

2. Setup transform in Glue Jobs

   • Select Source.
   • Select Amazon S3.
   • Enter your folder in raw bucket
   • Select your input data format.

   

   • Select your source.
   • Select Target.
   • Select Amazon S3.
   • Enter your folder in raw bucket
   • Select your output data format.

   

3. Setup detail of Glue Jobs

   • Enter your glue job name
   • Select your IAM glue role.
   • Enter number of worker
   • Select your Script path glue assets.
   • Select your Temporary path glue assets.
   • Select Save

   

   

   

4. Test Glue Job run

   • Select Run