o !dԜ@sddlZddlZddlZddlZddlZddlZddlZddlZddlmZ ddl m Z m Z ddl m Z mZddlmZddlZddlmZmZddlmZddlmZdd lmZeeZd Zd Zd d Z ddZ!ddZ"ddZ#ddZ$ddZ%   d,ddZ&Gddde'Z(Gddde)Z*Gddde*Z+Gd d!d!e*Z,Gd"d#d#e'Z-Gd$d%d%e'Z.Gd&d'd'e'Z/Gd(d)d)e'Z0Gd*d+d+eZ1dS)-N)error)datetime timedelta)tzparser) PyAsn1Error)get_trail_by_arnget_account_id_from_arn) BasicCommand) ClientError)ParameterRequiredErrorz%Y%m%dT%H%M%SZz%Y-%m-%dT%H:%M:%SZcC |tS)z;Returns a formatted date string in a CloudTrail date format)strftime DATE_FORMATdaterOusr/lib/python3.10/site-packages/awscli/customizations/cloudtrail/validation.py format_date( rcCr )z4Returns a formatted date string meant for CLI output)rDISPLAY_DATE_FORMATrrrrformat_display_date-rrcCs|jtdS)z.Returns a normalized date using a UTC timezone)tzinfo)replacerZtzutcrrrrnormalize_date2srcCs |ddS)zExtract the timestamp portion of a manifest file. Manifest file names take the following form: AWSLogs/{account}/CloudTrail-Digest/{region}/{ymd}/{account}_CloudTrail -Digest_{region}_{name}_region_{date}.json.gz iir)Z digest_s3_keyrrrextract_digest_key_date7s rcCs(zt|WStytd|w)NzUnable to parse date value: %s)rparse ValueError)Z date_stringrrr parse_dateAs    rcCs$td}||std|dS)zlEnsures that the arn looks correct. ARNs look like: arn:aws:cloudtrail:us-east-1:123456789012:trail/fooz$arn:.+:cloudtrail:.+:\d{12}:trail/.+zInvalid trail ARN provided: %sN)recompilematchr) trail_arnpatternrrrassert_cloudtrail_arn_is_validHs   r$c  Cst|d} |dur4t||} td| | d}| dd} | d} | r4| s,td|dd} |d d }|d d }| sHt|} t | ||||| d }t ||| |||t |dS)aCreates a CloudTrail DigestTraverser and its object graph. :type cloudtrail_client: botocore.client.CloudTrail :param cloudtrail_client: Client used to connect to CloudTrail :type organization_client: botocore.client.organizations :param organization_client: Client used to connect to Organizations :type s3_client_provider: S3ClientProvider :param s3_client_provider: Used to create Amazon S3 client per/region. :param trail_arn: CloudTrail trail ARN :param trail_source_region: The scanned region of a trail. :param on_invalid: Callback that is invoked when validating a digest fails. :param on_gap: Callback that is invoked when a digest has no link to the previous digest, but there are more digests to validate. This can happen when a trail is disabled for a period of time. :param on_missing: Callback that is invoked when a digest file has been deleted from Amazon S3 but is supposed to be present. :param bucket: Amazon S3 bucket of the trail if it is different than the bucket that is currently associated with the trail. :param prefix: bucket: Key prefix prepended to each digest and log placed in the Amazon S3 bucket if it is different than the prefix that is currently associated with the trail. :param account_id: The account id for which the digest files are validated. For normal trails this is the caller account, for organization trails it is the member accout. ``on_gap``, ``on_invalid``, and ``on_missing`` callbacks are invoked with the following named arguments: - ``bucket`: The next S3 bucket. - ``next_key``: (optional) Next digest key that was found in the bucket. - ``next_end_date``: (optional) End date of the next found digest. - ``last_key``: The last digest key that was found. - ``last_start_date``: (optional) Start date of last found digest. - ``message``: (optional) Message string about the notification. NzLoaded trail info: %sZ S3BucketNameZ S3KeyPrefixZIsOrganizationTrailzAMissing required parameter for organization trail: '--account-id'Z OrganizationZId:/) account_id trail_names3_client_providertrail_source_regiontrail_home_regionorganization_id)digest_providerstarting_bucketstarting_prefix on_invalidon_gap on_missingpublic_key_provider) r$rLOGdebuggetr Zdescribe_organizationsplitr DigestProviderDigestTraverserPublicKeyProvider)cloudtrail_clientorganization_clientr+r"r,r2r3r4bucketprefixr)r.Z trail_infoZ is_org_trailZ trail_regionr*r/rrrcreate_digest_traverserQsF(    rAc@s2eZdZdZd ddZddZddZd d Zd S) S3ClientProviderzCreates Amazon S3 clients and determines the region name of a client. This class will cache the location constraints of previously requested buckets and cache previously created clients for the same region. us-east-1cCs||_||_i|_i|_dSN)_session_get_bucket_location_region _client_cache _region_cache)selfsessionZget_bucket_location_regionrrr__init__s zS3ClientProvider.__init__cCs||}||S)z=Creates an S3 client that can work with the given bucket name)_get_bucket_region_create_client)rI bucket_name region_namerrr get_clients  zS3ClientProvider.get_clientcCsB||jvr||j}|j|d}|dpd}||j|<|j|S)zReturns the region of a bucket)BucketZLocationConstraintrC)rHrMrFZget_bucket_location)rIrNclientresultregionrrrrLs      z#S3ClientProvider._get_bucket_regioncCs,||jvr|jd|}||j|<|j|S)z5Creates an Amazon S3 client for the given region nameZs3)rGrE create_client)rIrOrRrrrrMs   zS3ClientProvider._create_clientN)rC)__name__ __module__ __qualname____doc__rKrPrLrMrrrrrBs   rBc@seZdZdZdS) DigestErrorz0Exception raised when a digest fails to validateN)rVrWrXrYrrrrrZsrZc eZdZdZfddZZS)DigestSignatureErrorz3Exception raised when a digest signature is invalidc d||f}tt||dS)Nz=Digest file s3://%s/%s INVALID: signature verification failed)superr\rKrIr?keymessage __class__rrrKszDigestSignatureError.__init__rVrWrXrYrK __classcell__rrrbrr\r\cr[)InvalidDigestFormatz4Exception raised when a digest has an invalid formatcr])Nz.Digest file s3://%s/%s INVALID: invalid format)r^rgrKr_rbrrrKszInvalidDigestFormat.__init__rdrrrbrrgrfrgc@ eZdZdZddZddZdS)r<z:Retrieves public keys from CloudTrail within a date range.cCs ||_dSrD)_cloudtrail_client)rIr=rrrrKs zPublicKeyProvider.__init__cCs6|jj||d}|d}td|tdd|DS)aLoads public keys in a date range into a returned dict. :type start_date: datetime :param start_date: Start date of a date range. :type end_date: datetime :param end_date: End date of a date range. :rtype: dict :return: Returns a dict where each key is the fingerprint of the public key, and each value is a dict of public key data. )Z StartTimeZEndTimeZ PublicKeyListzLoaded public keys in range: %scss|] }|d|fVqdS)Z FingerprintNr).0r`rrr sz4PublicKeyProvider.get_public_keys..)riZlist_public_keysr6r7dict)rI start_dateend_date public_keysZpublic_keys_in_rangerrrget_public_keyss   z!PublicKeyProvider.get_public_keysN)rVrWrXrYrKrprrrrr<s r<c@s>eZdZdZ  d ddZddZddZd d Zd d ZdS)r:a5 Retrieves digest keys and digests from Amazon S3. This class is responsible for determining the full list of digest files in a bucket and loading digests from the bucket into a JSON decoded dict. This class is not responsible for validation or iterating from one digest to the next. NcCs,||_||_||_||_|p||_||_dSrD)_client_providerr*r)r-r,r.)rIr+r)r*r-r,r.rrrrKs   zDigestProvider.__init__cCsg}|||}|j|}|d}|j||d} | d} tt|} tt|tdd} t | |} | D]}| |rUt |}|| krL|S|| krU||q:|S)aReturns a list of digest keys in the date range. This method uses a list_objects API call and provides a Marker parameter that is calculated based on the start_date provided. Amazon S3 then returns all keys in the bucket that start after the given key (non-inclusive). We then iterate over the keys until the date extracted from the yielded keys is greater than the given end_date. Z list_objects)rQMarkerzContents[*].Key)hours)_create_digest_keyrqrPZ get_paginatorZpaginatesearchrrrrr _create_digest_key_regexr!rappend)rIr?r@rmrndigestsmarkerrRZ paginatorZ page_iteratorZ key_filterZtarget_start_dateZtarget_end_dateZdigest_key_regexr`Zextracted_daterrrload_digest_keys_in_ranges*       z(DigestProvider.load_digest_keys_in_rangec Cs|j|}|j||d}zt|dtjdB}t| }Wnt t fy1t ||wd|dvs>d|dvrCt |||dd|d<|dd|d<||fS) zlLoads a digest by key from S3. Returns the JSON decode data and GZIP inflated raw content. rQKeyBody signatureZMetadatazsignature-algorithm _signatureZ_signature_algorithm)rqrP get_objectzlib decompressread MAX_WBITSjsonloadsdecoder ZLibErrorrgr\)rIr?r`rRrSdigest digest_datarrr fetch_digest#s"      zDigestProvider.fetch_digestcCsz|tdd}d}|jt||d|j|j|jd}|jr'|d7}|j|d<|d7}|jd i|}|r;|d |}|S) a~Computes an Amazon S3 key based on the provided data. The computed is what would have been placed in the S3 bucket if a log digest were created at a specific time. This computed key does not have to actually exist as it will only be used to as a Marker parameter in a list_objects call. :return: Returns a computed key as a string. rs)minutesAWSLogs/z%Y/%m/%d)r)rZymd source_region home_regionname{organization_id}/r.z{account_id}/CloudTrail-Digest/{source_region}/{ymd}/{account_id}_CloudTrail-Digest_{source_region}_{name}_{home_region}_{date}.json.gzr'Nr) rr)rrr,r-r*r.format)rIrm key_prefixrtemplatetemplate_paramsr`rrrru:s&   z!DigestProvider._create_digest_keycCsd}t|jt|jt|jt|jd}|jr%|d7}|j|d<|d7}|jd i|}|rrr=)rIrZ client_argsrrrrs&   z%CloudTrailValidateLogs.setup_servicesc Cst|j|j|j|j|j|j|j|j|j |j |j d }| | |j|j}|D],}|||jd7_|d|d|df|dsHq(|dD]}||qLq(|dS)N) r"r=r>r,r+r?r@r4r2r3r)rszDigest file s3://%s/%s validrrZlogFiles)rAr"r=r>rr+rr_on_missing_digest_on_invalid_digest_on_digest_gapr)_write_startup_textrrr_track_found_timesr _write_status _download_log_write_summary_text)rI traverserryrlogrrrrs2    zCloudTrailValidateLogs._callcCsDt|d}||jkr||_|js t|d}t||j|_dSdS)Nrr)rrrrminr)rIrZdigest_start_timeZdigest_end_timerrrrs   z)CloudTrailValidateLogs._track_found_timesc s4zi|j|d}|j|d|ddttjdB}t}tfdddD] }| |}| |q)| }|rA| || }||dkrS| |WdS|jd 7_|d |d|dfWdSty} z| jd d d kr{||WYd} ~ dSd} ~ wty||YdSw)z9 Download a log, decompress, and compare SHA256 checksumss3Buckets3Objectr|rcsddS)Nr~i)rrrrrsz6CloudTrailValidateLogs._download_log..Z hashValuerszLog file s3://%s/%s validrrrN)r+rPrr decompressobjrrriterrupdateflushr_on_log_invalidrrr r_on_missing_logr_on_invalid_log_format) rIrrRZ gzip_inflaterZ rolling_hashchunkdataZremaining_dataZ computed_hashrrrrrs8       z$CloudTrailValidateLogs._download_logFcCsZ|r|jrtjd|ntjd|d|_dS|jr+d|_tjd|dSdS)Nz%s z %s TFz%s )rsysstderrwriterstdout)rIraZis_errorrrrr/s z$CloudTrailValidateLogs._write_statuscCs(tjd|jt|jt|jfdS)Nz5Validating log files for trail %s between %s and %s )rrrr"rrrrIrrrr:s z*CloudTrailValidateLogs._write_startup_textcCs|js tjdtjdt|jt|jf|js'|js'tjddS|j r-|j s4tjdntjdt|j t|j f| |j|jd| |j |j dtjddS)N zResults requested for %s to %s zNo digests found z No valid digests found in range zResults found for %s to %s: rr)rrrrrrrrrrr _write_ratiorrr rrrr@s(    z*CloudTrailValidateLogs._write_summary_textcCsP||}|dkr$tjd|||f|dkr&tjd|||fdSdSdS)Nrz %d/%d %s files validz, %d/%d %s files INVALID)rrr)rIvalidinvalidrtotalrrrr Ss  z#CloudTrailValidateLogs._write_ratiocKs&|jd7_|d||fddS)Nrsz)Digest file s3://%s/%s INVALID: not foundTrr)rIr?rkwargsrrrr[s z)CloudTrailValidateLogs._on_missing_digestcKs(|dt|dt|dfddS)Nz;No log files were delivered by CloudTrail between %s and %srrT)rr)rIrrrrr`s  z%CloudTrailValidateLogs._on_digest_gapcKs|jd7_||ddS)NrsTr)rIrarrrrrfsz)CloudTrailValidateLogs._on_invalid_digestcC.|jd7_|d|d|dfddS)Nrsz+Log file s3://%s/%s INVALID: invalid formatrrTrrrIZlog_datarrrrjz-CloudTrailValidateLogs._on_invalid_log_formatcCr)Nrsz5Log file s3://%s/%s INVALID: hash value doesn't matchrrTrrrrrrprz&CloudTrailValidateLogs._on_log_invalidcCr)Nrsz&Log file s3://%s/%s INVALID: not foundrrTrrrrrrvrz&CloudTrailValidateLogs._on_missing_log)F)rVrWrXrYNAMEZ DESCRIPTIONZ ARG_TABLErKrrrrrrrrrr rrrrrrrerrrbrr]sX-   r)NNNNNNN)2rrrrloggingrrrrrrrZdateutilrrZ pyasn1.errorrrZ&awscli.customizations.cloudtrail.utilsrr Zawscli.customizations.commandsr Zbotocore.exceptionsr Z awscli.schemar getLoggerrVr6rrrrrrrr$rAobjectrBrrZr\rgr<r:r;rrrrrrsR          N#65