Class: AWSCDK::Glue::CfnDevEndpointProps

Inherits:
Jsii::Struct
  • Object
show all
Defined in:
glue/cfn_dev_endpoint_props.rb

Overview

Properties for defining a CfnDevEndpoint.

See Also:

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(role_arn:, arguments: nil, endpoint_name: nil, extra_jars_s3_path: nil, extra_python_libs_s3_path: nil, glue_version: nil, number_of_nodes: nil, number_of_workers: nil, public_key: nil, public_keys: nil, security_configuration: nil, security_group_ids: nil, subnet_id: nil, tags: nil, worker_type: nil) ⇒ CfnDevEndpointProps

Returns a new instance of CfnDevEndpointProps.

Parameters:

  • role_arn (String) —

    The Amazon Resource Name (ARN) of the IAM role used in this DevEndpoint .

  • arguments (Object, nil) (defaults to: nil) —

    A map of arguments used to configure the DevEndpoint .

  • endpoint_name (String, nil) (defaults to: nil) —

    The name of the DevEndpoint .

  • extra_jars_s3_path (String, nil) (defaults to: nil) —

    The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint .

  • extra_python_libs_s3_path (String, nil) (defaults to: nil) —

    The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint .

  • glue_version (String, nil) (defaults to: nil) —

    The AWS Glue version determines the versions of Apache Spark and Python that AWS Glue supports.

  • number_of_nodes (Numeric, nil) (defaults to: nil) —

    The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint .

  • number_of_workers (Numeric, nil) (defaults to: nil) —

    The number of workers of a defined workerType that are allocated to the development endpoint.

  • public_key (String, nil) (defaults to: nil) —

    The public key to be used by this DevEndpoint for authentication.

  • public_keys (Array<String>, nil) (defaults to: nil) —

    A list of public keys to be used by the DevEndpoints for authentication.

  • security_configuration (String, nil) (defaults to: nil) —

    The name of the SecurityConfiguration structure to be used with this DevEndpoint .

  • security_group_ids (Array<String>, nil) (defaults to: nil) —

    A list of security group identifiers used in this DevEndpoint .

  • subnet_id (String, nil) (defaults to: nil) —

    The subnet ID for this DevEndpoint .

  • tags (Object, nil) (defaults to: nil) —

    The tags to use with this DevEndpoint.

  • worker_type (String, nil) (defaults to: nil) —

    The type of predefined worker that is allocated to the development endpoint.



24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
 
# File 'glue/cfn_dev_endpoint_props.rb', line 24

def initialize(role_arn:, arguments: nil, endpoint_name: nil, extra_jars_s3_path: nil, extra_python_libs_s3_path: nil, glue_version: nil, number_of_nodes: nil, number_of_workers: nil, public_key: nil, public_keys: nil, security_configuration: nil, security_group_ids: nil, subnet_id: nil, tags: nil, worker_type: nil)
  @role_arn = role_arn
  Jsii::Type.check_type(@role_arn, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "roleArn")
  @arguments = arguments
  Jsii::Type.check_type(@arguments, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJhbnkifQ==")), "arguments") unless @arguments.nil?
  @endpoint_name = endpoint_name
  Jsii::Type.check_type(@endpoint_name, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "endpointName") unless @endpoint_name.nil?
  @extra_jars_s3_path = extra_jars_s3_path
  Jsii::Type.check_type(@extra_jars_s3_path, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "extraJarsS3Path") unless @extra_jars_s3_path.nil?
  @extra_python_libs_s3_path = extra_python_libs_s3_path
  Jsii::Type.check_type(@extra_python_libs_s3_path, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "extraPythonLibsS3Path") unless @extra_python_libs_s3_path.nil?
  @glue_version = glue_version
  Jsii::Type.check_type(@glue_version, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "glueVersion") unless @glue_version.nil?
  @number_of_nodes = number_of_nodes
  Jsii::Type.check_type(@number_of_nodes, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJudW1iZXIifQ==")), "numberOfNodes") unless @number_of_nodes.nil?
  @number_of_workers = number_of_workers
  Jsii::Type.check_type(@number_of_workers, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJudW1iZXIifQ==")), "numberOfWorkers") unless @number_of_workers.nil?
  @public_key = public_key
  Jsii::Type.check_type(@public_key, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "publicKey") unless @public_key.nil?
  @public_keys = public_keys
  Jsii::Type.check_type(@public_keys, JSON.parse(Base64.strict_decode64("eyJjb2xsZWN0aW9uIjp7ImVsZW1lbnR0eXBlIjp7InByaW1pdGl2ZSI6InN0cmluZyJ9LCJraW5kIjoiYXJyYXkifX0=")), "publicKeys") unless @public_keys.nil?
  @security_configuration = security_configuration
  Jsii::Type.check_type(@security_configuration, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "securityConfiguration") unless @security_configuration.nil?
  @security_group_ids = security_group_ids
  Jsii::Type.check_type(@security_group_ids, JSON.parse(Base64.strict_decode64("eyJjb2xsZWN0aW9uIjp7ImVsZW1lbnR0eXBlIjp7InByaW1pdGl2ZSI6InN0cmluZyJ9LCJraW5kIjoiYXJyYXkifX0=")), "securityGroupIds") unless @security_group_ids.nil?
  @subnet_id = subnet_id
  Jsii::Type.check_type(@subnet_id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "subnetId") unless @subnet_id.nil?
  @tags = tags
  Jsii::Type.check_type(@tags, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJhbnkifQ==")), "tags") unless @tags.nil?
  @worker_type = worker_type
  Jsii::Type.check_type(@worker_type, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "workerType") unless @worker_type.nil?
end

Instance Attribute Details

#arguments ⇒ Object? (readonly)

A map of arguments used to configure the DevEndpoint .

Valid arguments are:

  • "--enable-glue-datacatalog": ""
  • "GLUE_PYTHON_VERSION": "3"
  • "GLUE_PYTHON_VERSION": "2"

You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

Returns:

  • (Object, nil)

See Also:



74
75
76
 
# File 'glue/cfn_dev_endpoint_props.rb', line 74

def arguments
  @arguments
end

#endpoint_name ⇒ String? (readonly)

The name of the DevEndpoint .

Returns:

  • (String, nil)

See Also:



79
80
81
 
# File 'glue/cfn_dev_endpoint_props.rb', line 79

def endpoint_name
  @endpoint_name
end

#extra_jars_s3_path ⇒ String? (readonly)

The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint .

You can only use pure Java/Scala libraries with a DevEndpoint .

Returns:

  • (String, nil)

See Also:



86
87
88
 
# File 'glue/cfn_dev_endpoint_props.rb', line 86

def extra_jars_s3_path
  @extra_jars_s3_path
end

#extra_python_libs_s3_path ⇒ String? (readonly)

The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint .

Multiple values must be complete paths separated by a comma.

You can only use pure Python libraries with a DevEndpoint . Libraries that rely on C extensions, such as the pandas Python data analysis library, are not currently supported.

Returns:

  • (String, nil)

See Also:



95
96
97
 
# File 'glue/cfn_dev_endpoint_props.rb', line 95

def extra_python_libs_s3_path
  @extra_python_libs_s3_path
end

#glue_version ⇒ String? (readonly)

The AWS Glue version determines the versions of Apache Spark and Python that AWS Glue supports.

The Python version indicates the version supported for running your ETL scripts on development endpoints.

For more information about the available AWS Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

Development endpoints that are created without specifying a Glue version default to Glue 0.9.

You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

Returns:

  • (String, nil)

See Also:



108
109
110
 
# File 'glue/cfn_dev_endpoint_props.rb', line 108

def glue_version
  @glue_version
end

#number_of_nodes ⇒ Numeric? (readonly)

The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint .

Returns:

  • (Numeric, nil)

See Also:



113
114
115
 
# File 'glue/cfn_dev_endpoint_props.rb', line 113

def number_of_nodes
  @number_of_nodes
end

#number_of_workers ⇒ Numeric? (readonly)

The number of workers of a defined workerType that are allocated to the development endpoint.

The maximum number of workers you can define are 299 for G.1X , and 149 for G.2X .

Returns:

  • (Numeric, nil)

See Also:



120
121
122
 
# File 'glue/cfn_dev_endpoint_props.rb', line 120

def number_of_workers
  @number_of_workers
end

#public_key ⇒ String? (readonly)

The public key to be used by this DevEndpoint for authentication.

This attribute is provided for backward compatibility because the recommended attribute to use is public keys.

Returns:

  • (String, nil)

See Also:



127
128
129
 
# File 'glue/cfn_dev_endpoint_props.rb', line 127

def public_key
  @public_key
end

#public_keys ⇒ Array<String>? (readonly)

A list of public keys to be used by the DevEndpoints for authentication.

Using this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the UpdateDevEndpoint API operation with the public key content in the delete_public_keys attribute, and the list of new keys in the add_public_keys attribute.

Returns:

  • (Array<String>, nil)

See Also:



136
137
138
 
# File 'glue/cfn_dev_endpoint_props.rb', line 136

def public_keys
  @public_keys
end

#role_arn ⇒ String (readonly)

The Amazon Resource Name (ARN) of the IAM role used in this DevEndpoint .

Returns:

  • (String)

See Also:



61
62
63
 
# File 'glue/cfn_dev_endpoint_props.rb', line 61

def role_arn
  @role_arn
end

#security_configuration ⇒ String? (readonly)

The name of the SecurityConfiguration structure to be used with this DevEndpoint .

Returns:

  • (String, nil)

See Also:



141
142
143
 
# File 'glue/cfn_dev_endpoint_props.rb', line 141

def security_configuration
  @security_configuration
end

#security_group_ids ⇒ Array<String>? (readonly)

A list of security group identifiers used in this DevEndpoint .

Returns:

  • (Array<String>, nil)

See Also:



146
147
148
 
# File 'glue/cfn_dev_endpoint_props.rb', line 146

def security_group_ids
  @security_group_ids
end

#subnet_id ⇒ String? (readonly)

The subnet ID for this DevEndpoint .

Returns:

  • (String, nil)

See Also:



151
152
153
 
# File 'glue/cfn_dev_endpoint_props.rb', line 151

def subnet_id
  @subnet_id
end

#tags ⇒ Object? (readonly)

The tags to use with this DevEndpoint.

Returns:

  • (Object, nil)

See Also:



156
157
158
 
# File 'glue/cfn_dev_endpoint_props.rb', line 156

def tags
  @tags
end

#worker_type ⇒ String? (readonly)

The type of predefined worker that is allocated to the development endpoint.

Accepts a value of Standard, G.1X, or G.2X.

  • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
  • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
  • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

Returns:

  • (String, nil)

See Also:



169
170
171
 
# File 'glue/cfn_dev_endpoint_props.rb', line 169

def worker_type
  @worker_type
end

Class Method Details

.jsii_properties ⇒ Object 



171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
 
# File 'glue/cfn_dev_endpoint_props.rb', line 171

def self.jsii_properties
  {
    :role_arn => "roleArn",
    :arguments => "arguments",
    :endpoint_name => "endpointName",
    :extra_jars_s3_path => "extraJarsS3Path",
    :extra_python_libs_s3_path => "extraPythonLibsS3Path",
    :glue_version => "glueVersion",
    :number_of_nodes => "numberOfNodes",
    :number_of_workers => "numberOfWorkers",
    :public_key => "publicKey",
    :public_keys => "publicKeys",
    :security_configuration => "securityConfiguration",
    :security_group_ids => "securityGroupIds",
    :subnet_id => "subnetId",
    :tags => "tags",
    :worker_type => "workerType",
  }
end

Instance Method Details

#to_jsii ⇒ Object 



191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 
# File 'glue/cfn_dev_endpoint_props.rb', line 191

def to_jsii
  result = {}
  result.merge!({
    "roleArn" => @role_arn,
    "arguments" => @arguments,
    "endpointName" => @endpoint_name,
    "extraJarsS3Path" => @extra_jars_s3_path,
    "extraPythonLibsS3Path" => @extra_python_libs_s3_path,
    "glueVersion" => @glue_version,
    "numberOfNodes" => @number_of_nodes,
    "numberOfWorkers" => @number_of_workers,
    "publicKey" => @public_key,
    "publicKeys" => @public_keys,
    "securityConfiguration" => @security_configuration,
    "securityGroupIds" => @security_group_ids,
    "subnetId" => @subnet_id,
    "tags" => @tags,
    "workerType" => @worker_type,
  })
  result.compact
end