Untitled

 avatar
unknown
plain_text
2 years ago
1.0 kB
4
Indexable
write an checkDag function in python

Case 1.
a.First check DAG will check for the Airflow->Admin->Variable "odsBulkMigration_Interaction" and check for the value : {"override" : "Yes"}
b.If not case a then DAG will check for the "odsBulkMigration_Interaction" config file and check for the value override_flag=Yes.
If either of one paramter is set "Yes", then directly connect RDMS (Airflow->Connection->hobsdbhost) and fetch the data to store in cassandra "hobs_ods_staging"."hos_interaction"

Case 2.
Airflow->Admin->Variable "odsBulkMigration_Interaction" - Not Declared
[OR] Declared but value not provided (None)
[OR] No key value : {"override" : } or {"override" : ""}
[OR] Value : {"override" : "No"}
[OR] Config file "override_flag" not present
[OR} override_flag="" (None)
Either of all conditions failed and override_flag="No" in config file
Then would follow the "hos_migration_master" batch approach

Exception handling should be done at all stages, the code should not break for any unhandled exceptions.

 
Editor is loading...