이 페이지의 이전 버전을 보고 있습니다. 현재 버전 보기.

현재와 비교 페이지 이력 보기

« 이전 버전 21 다음 »

실습 주의사항 연결

https://bit.ly/2QCkHSp


실습 URL

Lab구분URL
Lab0사전작업https://www.awsdemokr.com/bigdata/hol/Lab0.pdf
Lab1Kinesishttps://www.awsdemokr.com/bigdata/hol/Lab1.pdf
Lab2Glue, Athena, QuickSighthttps://www.awsdemokr.com/bigdata/hol/Lab2.pdf
Lab3EMRhttps://www.awsdemokr.com/bigdata/hol/Lab3.pdf
Lab4Redshifthttps://www.awsdemokr.com/bigdata/hol/Lab4.pdf


Lab0. 주의사항: aws configure Error 대처 요령

ImportError: cannot import name AliasedEventEmitter

해결 방법

sudo yum downgrade aws-cli.noarch python27-botocore


Lab1. Kinesis 수정 사항

18페이지 아래 2번의 Format이 CSV가 아니라 JSON 입니다.


Lab3. EMR 실습 샘플 코드

https://github.com/setch3000/emr-s3



QuickSight 계정에 문제가 있을 때

qs.json
{
   "Version": "2012-10-17",
   "Statement": [
   {
     "Effect": "Allow",
     "Principal": {
        "Service": "quicksight.amazonaws.com"
     },
   "Action": "sts:AssumeRole"
   }
 ]
}

AWS CLI command

aws iam create-role --role-name aws-quicksight-service-role-v0 --assume-role-policy-document file://qs.json --path \/service-role/

aws iam attach-role-policy --role-name aws-quicksight-service-role-v0 --policy-arn arn:aws:iam::aws:policy/service-role/AWSQuicksightAthenaAccess
aws iam attach-role-policy --role-name aws-quicksight-service-role-v0 --policy-arn arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly
aws iam attach-role-policy --role-name aws-quicksight-service-role-v0 --policy-arn arn:aws:iam::{AccountID}:policy/service-role/AWSQuickSightIAMPolicy
aws iam attach-role-policy --role-name aws-quicksight-service-role-v0 --policy-arn arn:aws:iam::{AccountID}:policy/service-role/AWSQuickSightRDSPolicy
aws iam attach-role-policy --role-name aws-quicksight-service-role-v0 --policy-arn arn:aws:iam::{AccountID}:policy/service-role/AWSQuickSightRedshiftPolicy
aws iam attach-role-policy --role-name aws-quicksight-service-role-v0 --policy-arn arn:aws:iam::{AccountID}:policy/service-role/AWSQuickSightS3Policy



Lab 4. Redshift

Page9. PDF에서 코드 가져올 때, 에러 발생 할 수 있어서 제공

CREATE EXTERNAL TABLE spectrum.sales(
 salesid INTEGER,
 listid INTEGER,
 sellerid INTEGER,
 buyerid INTEGER,
 eventid INTEGER,
 dateid SMALLINT,
 qtysold SMALLINT,
 pricepaid DECIMAL(8,2),
 commission DECIMAL(8,2),
 saletime TIMESTAMP
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE
LOCATION 's3://id-redshift-uswest2/tickit/spectrum/sales/'
TABLE PROPERTIES ('numRows'='172000')


Page12. PDF에소 코드 가져올 때, 에러 발생할 수 있어서 제공

COPY event
FROM 's3://id-redshift-uswest2/tickit/allevents_pipe.txt'
IAM_ROLE 'ROLE_ARN으로 바꿔 넣으세요.'
DELIMITER '|'
TIMEFORMAT 'YYYY-MM-DD HH:MI:SS'
REGION 'us-west-2'



참고 자료

Kinesis Agent를 이용해서 Data firehose로 데이터 전송

https://docs.aws.amazon.com/ko_kr/firehose/latest/dev/writing-with-agents.html



  • 레이블 없음