Installation¶
Firstly, Docker is an easy-to-install application for your Mac or Windows environment that can build and run packaged applications in containers and microservices on virtually any platform. In order to access and install the inforion docker image, so its availabe to use, please insert in your Terminal (MacOs) or Command Prompt (Windows):
docker run fellowconsulting/inforion
In any case, here you can access this link for more information about the inforion docker image:
fellowconsulting/inforion Dockerhub
A modern, web-based UI is provided to facilitate use, in which the various data flows (transformation to load) can also be scheduled. This is the Prefect.io UI where we display the Workflows. Here is a quick look:
Docker Desktop¶
Another important point, is to have downloaded Docker Desktop, which depending on your working environment, Mac or Windows, you can download it from here:
The important part about the Docker Desktop is that it allows you to access the required files to start running the prefect flows. That is why, firstly, you need to have access to the following files mentioned in “Prerequisites”.
Prerequisites¶
But, before you start, it is important that you check if you have created a conf directory. These are necessary files to make test loads, as well as for the data extraction, trasnformation and load. So, In the directory you should have the following files:
Mapping_Kundenstamm.xlsx
credentials.yml: Here you define the access to M3, the database with the respective data, the Prefect.io and Great Expectation environment.
IONAPI Key File: Here the access to the APIs is provided.
module_dependencies.csv: Here the dependencies between different objects are defined: Module_name and Parent_module. These are also displayed in the UI.
modules.csv: Here parameters are defined that are used to display and run the dataflows in the UI.
Another important prerequisite is that we add the conf folder (as seen above) as well as the Great Expectations folder to the Docker Desktop. It is as simple as opneing the Docker and getting into Resources, and next File Sharing, to add the new directories. When this is done, you can actually start testing your data loads with Prefect flows.
Just to finish this segment, here are some common and very important commands when starting and upgrading the inforion Docker image.
Start on Mac or Linux¶
docker run -v $PWD/conf:/ion-workflow/conf fellowconsulting/inforion flow -b /ion-workflow/conf -c credentials.yml -m modules.csv -d module_dependencies.csv
Start on Windows¶
docker run -v %cd%/conf:/ion-workflow/conf fellowconsulting/inforion flow -b /ion-workflow/conf -c credentials.yml -m modules.csv -d module_dependencies.csv
Upgrade to Latest Build¶
docker pull fellowconsulting/inforion:latest