![]() Otherwise, the default DatahubClientConfig. Default: The datahub_api config if set at pipeline level. ![]() "description" : "The configuration required for initializing the state provider. "description" : "The type of the state provider to use. "title" : "DynamicTypedStateProviderConfig" , "description" : "Regex patterns for databases to filter in ingestion." , Takes precedence over other connection parameters." , "description" : "URI of database to connect to. "description" : " Alias to apply to database when ingesting." , "$ref" : "#/definitions/GEProfilingConfig" "profile_table_row_count_estimate_only" : false , "max_number_of_fields_to_profile" : null , "include_field_distinct_value_frequencies" : false , "turn_off_expensive_profiling_metrics" : false , "description" : "Whether to use a file backed cache for the view definitions." , Requires `include_view_lineage` to be enabled." , "description" : "Populates column-level lineage for view->view and table->view lineage using DataHub's sql parser. "title" : "Include View Column Lineage" , "description" : "Populates view->view and table->view lineage using DataHub's sql parser." , "description" : "If the source supports it, include table lineage to the underlying storage location." , "title" : "Include Table Location Lineage" , "description" : "Whether tables should be ingested." , "description" : "Whether views should be ingested." , There can be multiple domain keys specified." , Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like \"Marketing\".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. "description" : "Attach domains to databases, schemas or tables during ingestion using regex patterns. Note that only tables allowed by the `table_pattern` will be considered." , "description" : "Regex patterns to filter tables (or specific columns) for profiling during ingestion. to match all views starting with customer in Customer database and public schema, use the regex '.*'" , Specify regex to match the entire view name in format. Note: Defaults to table_pattern if not specified. "description" : "Regex patterns for views to filter in ingestion. to match all tables starting with customer in Customer database and public schema, use the regex '.*'" , Specify regex to match the entire table name in format. ![]() "description" : "Regex patterns for tables to filter in ingestion. "$ref" : "#/definitions/AllowDenyPattern" "description" : "Deprecated in favour of database_pattern." , To set connection arguments in the URL, specify them under `connect_args`." , "description" : "Any options specified here will be passed to () as kwargs. "$ref" : "#/definitions/StatefulStaleMetadataRemovalConfig" "description" : "The instance of the platform that all assets produced by this recipe belong to" , "description" : "The environment that all assets produced by this connector belong to" , "description" : "Whether to convert dataset urns to lowercase." , This can affect processing time but can protect from sql parser's mem leak." , "description" : "When enabled, sql parser will run in isolated in a separate process. "title" : "Sql Parser Use External Process" , When disabled, re-states lineage on each run." , "description" : "When enabled, emits lineage as incremental to existing lineage already in DataHub. "description" : "Base configuration class for stateful ingestion for source configs to inherit from." ,
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |