How To Use Geocoding for HansaWorld with TomTom Integration

Enabling Geocoding Feature

Geocoding feature is used to convert customer/supplier address data into geographical coordinates that can later be used for data visualization on a Map chart. This feature can be used in any HansaWorld data cube for which customer (CUVc) register is imported.

1. Create a TomTom Account

For the geocoding feature to work, you have to create an account with TomTom (https://developer.tomtom.com/user/register) and get a consumer API key for the Search app. Please check the TomTom documentation regarding the number of free API calls per day and the information about purchasing additional credit for the TomTom service.

2. Enable Geocoding (For Enterprise Users)

If you are an Enterprise user, you should enable geocoding according to the following instructions, but if you are a Cloud user, please, contact us at support@flex.bi and we will do it for you.

Geocoding is used to link customer addresses to longitude and latitude values. 

• Add the following code to your config/eazybi.toml file:

  CODE

  [[regular_jobs]]
  job = "Geocoding.perform"
  every = "24h"

  Specify a new regular job Geocoding.perform and the frequency for the execution of this job. In the code example below, a frequency of 24h is set. You can specify a value that corresponds to your needs.
  
  

• Add the following code to the [general] section of your config/eazybi.toml file:
  

  CODE

  tomtom_api_key = "<your_tomtom_api_key>"
  tomtom_base_URL = "api.tomtom.com"
  tomtom_version_number = "2"

  
  In the code block above, you have to replace <your_tomtom_api_key> with your actual TomTom API key. Also, you have to check if the defined version is the current version of the API.
  
  

• Define the following plan parameters in the Update Account section's Plan parameters box of your flex.bi account:
  

  CODE

  enable_geocoding = true
  min_address_precision = 8
  min_address_similarity = 0.85

  You can also use the following plan parameter, to enable debugging mode and log geocoding errors:

  CODE

  debug_geocoding = "error"

• min_address_precision is the minimum geocoding result precision that is required to save the latitude and longitude values. The recommended value is 8, but you can increase it, to make the rules for result acceptance more strict, or decrease it, to make the rules less strict. The precision is a value returned by the TomTom service together with the geocoding results, which specifies how precise are the returned results based on the TomTom algorithm.

• min_address_similarity is the minimum similarity between the original address and the address returned from the geocoding service. The recommended value is 0.85, but you can increase it, to make the similarity rules more strict, or decrease it, to make the rules less strict. The similarity is a value calculated by comparing the address saved in flex.bi and the results returned from the TomTom geocoding service.

If one of the mentioned minimal values are not met by the result received from the TomTom geocoding service, it is deemed to be too imprecise and is not saved in flex.bi. The affected customer/supplier record is assigned a value of -1 for the property Geocoding needed and is skipped during the next geocoding runs, until the customer/supplier information is altered, providing more address details. Also the address and precision returned from the geocoding service is saved in Address Precision and Geocoding address properties, respectively, to help you determine if you should adjust the address information in flex.bi or the minimum precision or similarity parameters, to get better geocoding results. The calculated similarity is saved in the Address similarity property. To learn more about these limits and adjusting your address data for better geocoding results, see Troubleshooting section below. 

3. Import Your Data

If you haven't imported any data yet, proceed with data import.

If you have already imported data, make sure that you have imported the customer register (CUVc). If you haven't done that, import this register. After importing customer register, geocoding will be run automatically according to the schedule you set in the step 2.

4. Select Calculated Members For Latitude and Longitude

In this example, we will use the Customer dimension, but you can also do this for the Supplier dimension.

Select Customer in rows and Measures in columns. From Calculated members section select Customer latitude and Customer longitude under Customer properties. These measures should be the first selected measures in precisely this order - latitude, longitude.

If these calculated members are missing, you can create them, by clicking on Define new and using the following code:

 Customer latitude:

[Customer].CurrentMember.getProperty('Lat')

and

Customer longitude :

[Customer].CurrentMember.getProperty('Lng')

OR

Supplier latitude:

[Supplier].CurrentMember.getProperty('Lat')

and

Supplier longitude:

[Supplier].CurrentMember.getProperty('Lng')

5. Select Measures To Be Shown Your Map Chart

Select all other measures that you want to show on a map.

6. Select OpenStreetMap Or Wikimedia Map

Go to Map tab and select OpenStreetMap or Wikimedia map.

By hovering over a point on the map, you can see the selected measures for this location.

Troubleshooting

If you are getting an errorDeveloper Over Qps, it means that your geocoding request frequency is too high. To reduce request frequency, set a sleep interval between the requests, using the following parameter in the eazybi.toml file. The default value is 2 seconds:

sleep_length = 2

If no geocoding results are returned for a customer/supplier or the results are below the minimum precision and similarity limits:

• No geocoding data is stored for this customer/supplier in properties Customer latitude and Customer longitude (or Supplier latitude and Supplier longitude);
• The customer/supplier it is marked using Geocoding needed property value -1;
  • value -1 means that there was an issue with the particular customer/supplier and you have to do something about it;
  • value 1 means that this customer/supplier is scheduled for geocoding;
  • value 0 means that geocoding for this customer/supplier was successful and latitude and longitude data is stored in the system. 
• For troubleshooting purposes, the resulting address is stored in the Geocoding address property;
• For troubleshooting purposes, the resulting address precision is stored in the Address precession property;
• For troubleshooting purposes, the resulting address similarity is stored in the Address similarity property.

To get the missing geocoding data, you have to check the previously mentioned properties and the customer/supplier properties used for geocoding (street, city, inv_addr2, inv_addr3, inv_addr4, country_code) and make the necessary changes either in the precision and similarity limits or customer/supplier address data.

For analyzing geocoding address precision and similarity settings and the quality of the address data in HansaWorld, you can use special reports, like the ones we have included in our Templates library:

Geocoding - Bad Address Data

Geocoding - Good Address Data

To include these reports in your flex.bi account, just export report definitions from our Templates library and import them in your account (see instructions in How to export/import report and dashboard definitions).

Adjusting Minimum Address Precision and Similarity Parameters

If, after comparing the geocoding result stored in the Geocoding address property and customer/supplier address data stored in properties street, city, geocoding_needed, inv_addr2, inv_addr3, inv_addr4, country_code, you feel that the geocoding result is good enough to be stored in the system, you can lower the minimum address precision and similarity limits (see step 2) to match the values in properties Address precession and Address similarity. After changing these values in the Update Account section's Plan parameters box, all you have to do is wait for the next scheduled geocoding run when the latitude and longitude values will be fetched for the particular customer/supplier.

Likewise, if you feel that the geocoding results stored are not precise enough, you can increase the minimum address precision and similarity limits (see step 2).

Improving Customer/Supplier Address Data In HansaWorld

If no geocoding results are returned for a particular customer/supplier or the results are not precise enough, this can be caused by missing, incomplete or wrong address data in HansaWorld. As mentioned before, Geocoding uses all of the following customer/supplier properties: street, city, inv_addr2, inv_addr3, inv_addr4, country_code, so these are the values that you should check and improve if necessary. The following are some basic tips for checking and improving address data in HansaWorld:

• Check if the complete address data (street + city + inv_addr2 + inv_addr3 + inv_addr4 + country_code) gives a precise result in a maps search engine like Google Maps. If not, try to add/omit something, until you get the result you need.
• Make sure that the complete address data constitutes a full address containing country, street name, building number, zip code, region name, district name etc.
• Remove any information not related to the address, for example, name, phone number etc.
• Remove any information that you would not give to a taxi driver, like the name of a building, floor number, name of an office etc.

After fixing some address data in HansaWorld, you have to reimport this data in flex.bi and wait for the next scheduled geocoding run, for the changes to take effect.