Data Quality

1. Overview

The purpose of this script is to identify the data quality provided by a broker  over a specific time-frame.

2. Who is this script for?

Any trader who would like to investigate the data quality of their broker's price data can use this script. The script will work for any symbol and regulated broker using the MT5 platform. Data quality is specifically important for automated traders which backtest strategies that require high quality datasets.

3. What does this script do?

This script completes the following steps:

• Checks the date inputs set by the user
• startDate: the date from which the data should be checked.
• endDate: the date to which the data should be checked.
  
• Downloads the price data for the specified date inputs
• Establishes the quoting hours of the symbol based on the broker's specifications
• Checks the data quality by identifying missing data-points which exist within the broker's data-set
• Establishes the sequences of missing data-points
• Exports the results as a CSV file into the root data folder at the following location: MQL5/Files/PxStrat/Data Quality
  

4. Why is data quality important?

MT5 brokers provide historical price data of symbols for free. The data is downloaded from the broker's server and saved as M1 bars within the root folder; for example, EURUSD data from the broker ActivTrades would be saved at the following location of the root data folder: basesActivTrades-ServerhistoryEURUSD.

Due to how MT5 saves the historical price data, it is difficult to easily identify missing data points. Missing data-points arise if no tick (price change) event occurs during any 1 minute period. Hence, even if a symbol changes price in the real markets, if the broker does not update the price on the server, it will not be updated on the MT5 platform. Consequently, if no tick events occurs for an entire M1 period, then that data point will be missing within the history.

This has implications for any M1 based trading strategies, for example, a 10-minute MA would in fact reference the price of the last 11 minutes, and thereby distort the real value of the MA. Additionally, the number of missing data points can be used as a proxy for the quality and trustworthiness of a broker.

5. Assumptions and Limitations

At PxStrat, we try to be as transparent with the assumptions and limitations of our scripts as possible:

• Quoting hours are established by referring to the symbol specification page provided by the broker. It is assumed that the hours listed by the broker are consistent for the entire price history. This is true for most of the weeks of the year, except for:
  
• Public holidays (25th of December and 1st of January, etc.)
• DST changes (normally in late March and October)
  
• The script assumes that no more than 10 individual sessions occur on a daily basis. This is unlikely to be exceeded except for highly unique financial products.

Please let us know how satisfied you are with this script, by either leaving a comment here or at our website at PxStrat.

Thank you,

The PxStrat Team

nclick="viewGallery_GalleryManager.GoToPrev();"> nclick="viewGallery_GalleryManager.GoToPrev();">

ta itemprop="image" content="https://c.mql5.com/31/349/data-quality-screen-5337.PNG" />

nclick="viewGallery_GalleryManager.GoTonext();"> nclick="viewGallery_GalleryManager.GoTonext();">