Class: AWSCDK::Glue::CfnCrawler::HudiTargetProperty

Inherits:
Jsii::Struct
  • Object
show all
Defined in:
glue/cfn_crawler.rb

Overview

Specifies an Apache Hudi data source.

See Also:

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(connection_name: nil, exclusions: nil, maximum_traversal_depth: nil, paths: nil) ⇒ HudiTargetProperty

Returns a new instance of HudiTargetProperty.

Parameters:

  • connection_name (String, nil) (defaults to: nil) —

    The name of the connection to use to connect to the Hudi target.

  • exclusions (Array<String>, nil) (defaults to: nil) —

    A list of glob patterns used to exclude from the crawl.

  • maximum_traversal_depth (Numeric, nil) (defaults to: nil) —

    The maximum depth of Amazon S3 paths that the crawler can traverse to discover the Hudi metadata folder in your Amazon S3 path.

  • paths (Array<String>, nil) (defaults to: nil) —

    An array of Amazon S3 location strings for Hudi, each indicating the root folder with which the metadata files for a Hudi table resides.



901
902
903
904
905
906
907
908
909
910
 
# File 'glue/cfn_crawler.rb', line 901

def initialize(connection_name: nil, exclusions: nil, maximum_traversal_depth: nil, paths: nil)
  @connection_name = connection_name
  Jsii::Type.check_type(@connection_name, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "connectionName") unless @connection_name.nil?
  @exclusions = exclusions
  Jsii::Type.check_type(@exclusions, JSON.parse(Base64.strict_decode64("eyJjb2xsZWN0aW9uIjp7ImVsZW1lbnR0eXBlIjp7InByaW1pdGl2ZSI6InN0cmluZyJ9LCJraW5kIjoiYXJyYXkifX0=")), "exclusions") unless @exclusions.nil?
  @maximum_traversal_depth = maximum_traversal_depth
  Jsii::Type.check_type(@maximum_traversal_depth, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJudW1iZXIifQ==")), "maximumTraversalDepth") unless @maximum_traversal_depth.nil?
  @paths = paths
  Jsii::Type.check_type(@paths, JSON.parse(Base64.strict_decode64("eyJjb2xsZWN0aW9uIjp7ImVsZW1lbnR0eXBlIjp7InByaW1pdGl2ZSI6InN0cmluZyJ9LCJraW5kIjoiYXJyYXkifX0=")), "paths") unless @paths.nil?
end

Instance Attribute Details

#connection_name ⇒ String? (readonly)

The name of the connection to use to connect to the Hudi target.

If your Hudi files are stored in buckets that require VPC authorization, you can set their connection properties here.

Returns:

  • (String, nil)

See Also:



918
919
920
 
# File 'glue/cfn_crawler.rb', line 918

def connection_name
  @connection_name
end

#exclusions ⇒ Array<String>? (readonly)

A list of glob patterns used to exclude from the crawl.

For more information, see Catalog Tables with a Crawler .

Returns:

  • (Array<String>, nil)

See Also:



925
926
927
 
# File 'glue/cfn_crawler.rb', line 925

def exclusions
  @exclusions
end

#maximum_traversal_depth ⇒ Numeric? (readonly)

The maximum depth of Amazon S3 paths that the crawler can traverse to discover the Hudi metadata folder in your Amazon S3 path.

Used to limit the crawler run time.

Returns:

  • (Numeric, nil)

See Also:



932
933
934
 
# File 'glue/cfn_crawler.rb', line 932

def maximum_traversal_depth
  @maximum_traversal_depth
end

#paths ⇒ Array<String>? (readonly)

An array of Amazon S3 location strings for Hudi, each indicating the root folder with which the metadata files for a Hudi table resides.

The Hudi folder may be located in a child folder of the root folder.

The crawler will scan all folders underneath a path for a Hudi folder.

Returns:

  • (Array<String>, nil)

See Also:



941
942
943
 
# File 'glue/cfn_crawler.rb', line 941

def paths
  @paths
end

Class Method Details

.jsii_properties ⇒ Object 



943
944
945
946
947
948
949
950
 
# File 'glue/cfn_crawler.rb', line 943

def self.jsii_properties
  {
    :connection_name => "connectionName",
    :exclusions => "exclusions",
    :maximum_traversal_depth => "maximumTraversalDepth",
    :paths => "paths",
  }
end

Instance Method Details

#to_jsii ⇒ Object 



952
953
954
955
956
957
958
959
960
961
 
# File 'glue/cfn_crawler.rb', line 952

def to_jsii
  result = {}
  result.merge!({
    "connectionName" => @connection_name,
    "exclusions" => @exclusions,
    "maximumTraversalDepth" => @maximum_traversal_depth,
    "paths" => @paths,
  })
  result.compact
end