Problems with Conversion to FXT and Tick Data Usage in Tick Data Suite [Outdated]

Converting to FXT

That can only be due to the script trying to open a non existing file so either the CSV is being copied to the wrong folder or the named is incorrect. Check the following items:

A hex editor (e.g. WinHex, Neo Hex Editor) can be used to change it at offset 0xFC. Be aware that it's going to be displayed in hexadecimal so a calculator may have to be used here.

Look up the offset in FXTHeader.mqh, which is displayed next to each setting. Use a hex editor. Note that all values are in little Indian format, meaning that if a value that exceeds one byte it has to be filled in the bytes to its right, e.g. to write 300 in the file it will have to be written as 2C 01.

This is common with Windows 7 / Windows Vista when running MT4 from Program Files folder. The issue is as the UAC is enabled and these operating systems use folder virtualization. In Windows 7, the resulting files can be found in c:\ProgramData\ typically, while in Windows Vista they can be found in c:\Users\username\AppData\Local\VirtualStore\Program Files\. If they're not there, just search for *.FXT, they have to be somewhere. To work around this issue, either copy the MT4 folder in a location not protected by UAC (e.g. the Desktop) or simply disable UAC; by typing UAC in the start->search box thing and follow the on-screen steps.

This might be due to the lack of a MIN_LOT and LOT_STEP for the symbol in the resulted FXT file. This occurs when creating the FXT file with a MT4 client while not connected to the broker on the script launching. As mentioned, the terminal should be connected to the broker on generating an FXT file.

This is common due to either failing to select a delimiter or selecting a dot (.) as the delimiter, so the solution is simply by going back to JForex, selecting comma (,) as the delimiter and exporting the CSV once again. This should be much faster than the first time as the data is now cached. You may take another look at the JForex Downloading Guide, for more details about other parameters.

No.

Yes. Duplicate information and even the JForex header line will be disregarded by the CSV2FXT script if it is encountered in the middle of the CSV file.

If JForex is involved, the CSV file start time must be from when the older CSV file ends. The new CSV file header row can be deleted before concatenating, even this step can be skipped while the CSV2FXT script will skip that line and reveals a warning concerning that in the experts log. By finishing the export, simply append the new CSV to the older one.

If the PHP scripts are involved, stopping the processing at a month end is perfect. The processing script automatically appends to an existing file so by resuming from the start of the next month the CSV will be perfect. Yet, the processing could be stopped even during the month; for instance, if a file is named EURUSD.csv and ends on 19.03.2012, by appending data starting from 01.03.2013 and ending with 02.04.2013 (e.g. by typing php process_dukascopy_data.php EURUSD 201303 201305 EURUSD.csv), the new data will be appended to the existing CSV and the data between 01.03 and 19.03 will be in the CSV twice. In which case, about 20 errors will be added by the CSV2FXT script will to the log and an alert about older ticks will be expelled, but it is more logical to skip the duplicated data so the resulting FXT should be fully consistent, although it will take a little bit longer while skipping the duplicated period.

It's mostly due to exporting the CSV file by JForex with failure to select the comma (,) as the field delimiter as mentioned in the guide. As for some reason that grasping the default JForex setting is to use a space as field separator and put several commas at the end of each line, the CSV file is useless and has to be regenerated. Fortunately the JForex caches the data so there is no need to wait for the download once more.

Two steps:

It should be noted if this change is made, that the bar count displayed in the results will be wrong – it could be changed too but there isn't much point as it has no effect on the backtest.

No.

• When running CSV2FXT, the message "Could not open input file". How can this be solved?
  • The CSV file is in experts\files.
  • The CSV file is in the correct MT4 installation folder (if you have more than one).
  • If CsvFile field is left blank, ensure that file name is the same as the symbol and with a csv extension.
  • If the CsvFile is updated, ensure that you the full file name is typed including its extension.
  • If your system is adjusted to hide the file extension for known files, your file might be named EURUSD.CSV.CSV and shown by the explorer as just EURUSD.CSV, or your file might be correctly named EURUSD.CSV and the explorer might just show EURUSD. In Windows 7, to try to overcome this problem you can hit ALT in an explorer window, click Tools -> Folder options -> View then uncheck "Hide extensions for known file types". This is the same for other Windows versions.
  • If Windows Vista is the running system and the MT4 installation is in Program Files, a copy should be made somewhere else (e.g. on the Desktop) and the work should be from that one instead. UAC in Vista is a security behavior that could be a real obstacle throughout the process, it can also be disabled, continue reading to know how.
• How could the fixed spread in an existing FXT file be changed?
• How could [some other setting] be changed in an existing FXT file?
• The resulting FXT and HST files are not there. Where are they?!
• Backtesting an EA with a newly created FXT causes Order Send Error 131. Why?
• If an alert saying "Bad CSV Format" appears while trying to convert the CSV exported from JForex, What to do?
• Could tick data be appended to an existing FXT file (and the accompanying HST files)?
• Could tick data be appended to an existing CSV file?
• Why does the CSV2FXT script get stuck once it is started? The progress indicator is not displayed on the chart and the only message displayed in the experts log is "The date column appears to be 0. Sample ..."
• How could I change an existing FXT file timeframe?
  • As the file name contains the timeframe, for example EURUSD15_0.FXT for an M15 EURUSD FXT file, the file has to be renamed (or copied) for the needed timeframe. Assuming it'd be H4, it is translated to minutes; 4 hours = 240 minutes and the file is renamed to EURUSD240_0.FXT.
  • Next a hex editor is used to edit the file at offset 0xD0. The number is changed to the same one used in the file name in the previous step (but be certain that it's converted to hex; using the example above, 240 is 0xF0 so it would have to be changed from 0x0F – that is 15 – to 0xF0).
• Could range bars or Renko bars be created with CSV2FXT?

Using the Tick Data

The answer is really the same as for question #2 above.

This means that MT4's refuses to overwrite an FXT file which is set read-only.

Because the FXT being used was created with an older script. It doesn't matter, even you can safely ignore it.

This means that the obtained results were not profitable.However, if you want to see them, select the Optimization Results tab, right click inside it and deselect "Skip Useless Results".

This occurs just when backtesting EAs that need the Metatrader 4 terminal to be connected to the broker such as Wallstreet Forex Robot or FAP Turbo. Simply, as the FXT is already there, the backtest starts too fast. This can be solved only by adding an artificial delay through the Configuration program installed with the Tick Data Suite, by increasing the backtest delay factor to something like 3. This will give a sensible enough delay with the backtest start for the terminal to be connected to the broker. If not resolved as such increase it to something like 10, and once it's done with that particular EA set it back to 0 unless you want to keep a permanent delay on backtesting or optimizing with tick data. Be aware that this will have no effect on already running Metatrader 4 terminals, so the need to be started first to take advantage of this option.

This is as real spread is used on creating the FXT. Enabling this option will cause the spread to be stored in the volume field. If the strategy makes use of the volume number, simply the number of ticks in MT4, fixed spread is needed to be used or other strategies such as counting the number of ticks and storing it in an array should be employed.

Two reasons are possible:

This is a typically occurring issue due to using the Birt's patch script and in this condition two possible causes are suggested:

If none of the above is the cause, you should first try to backtest the MACD EA using the same FXT, if stopped at the same point, there might be an issue in that range of the source CSV and the log of the CSV2FXT script could give you a description of any potential errors.

This just occurs when a start / end date is being used for the optimization. Unfortunately, this is a bug in Metatrader 4; only the first pass of an optimization uses the selected start and end date; subsequent runs use the available full range in the FXT.

When using the parameters for one of the optimization results, and to get the same outcome when backtesting, an FXT spanning exactly the period that needed to be optimized for can be used, or alternatively Use date can be simply be disabled when running the backtest for the selected optimization results; if it's not the parameters for the first run, that is.

• Why are the FXT files being overwritten?
  • If the free script is being used:
    • The Birt's patch script didn't run, it must be run whenever the MT4 terminal restarts while backtesting with tick data.
    • DLL imports are not enabled (Tools->Options->Expert advisors).
  • If the Tick Data Suite is being used, the MT4 terminal didn't start via tds.exe, and there are probably disabled DLLs, so the CSV2FXT script failed to set the file read-only.
• Why does terminal creates an FXT file when backtesting starts?
  • Be sure that the FXT file was created for the currency pair and timeframe that will be used in backtest?
  • If it's already created, then ensure that it was copied to the correct location, including the correct MT4 installation folder if there are more than one.
  • Also check question #1 answer above.
• Backtest runs but the modeling quality isn't 99%. Why?
• Backtesting stops once the start button clicked. The backtesting journal tab displays "TestGenerator: internal error because the file not opened". What is wrong?
  • If Birt's Patch script is involved, you may have forgot to run it as you must run it whenever the terminal restart, before starting the backtest, or an outdated version of the script may be in use.
  • If the Tick Data Suite is involved, you didn't launch the terminal via tds.exe.
• Why does the terminal crash once "start" is pressed in the backtest user interface?
  • If a build other than latest MT4 build is being used, this is mostly as the EA was compiled with a more recent build than the one being used for backtesting and Metaquotes didn't think it wise to add ex4 versioning. This can be ascertained after attaching the same EA to a chart, if a crash results. Using the latest build or having the EA compiled with build being used for backtesting will solve the issue.
  • If the free script is being used, ensure that a backtest was run prior to running the script.
  • If the free script is involved, optimizations will not run.
• Why a red bar appears at the end of the backtest despite the modeling quality is 99%?
• What could be done if the optimization results window is empty despite MT4 seeming to run multiple passes fine?
• The results are empty (no trades) in all the optimization passes when using the Walk Forward Analyzer. Is there a solution?
• Why are all the volumes in tick data backtest smaller than 1?
• Why does the backtest fails on attempting to start it with the backtest journal saying "No data for testing."?
  • Use date is enabled while the selected start and end dates are outside the range covered by the FXT.
  • An FXT larger than 4GB is being used with the Birt's patch script or an FXT larger than 2GB without limitation removing, Use date is enabled while the selected start and end dates are above the 4GB range in the file (respectively above 2GB). Tick Data Suite enabled the use of FXT files up to 208 GB in size.
• The backtest stops somewhere in the middle, despite the created FXT file spans a longer time period. How can this be fixed?
  • A file larger than 4GB is being used; by reaching the 4GB limit the backtest will stop. This might also be due to the use of a file larger than 2GB while the limitation removal was not enabled.
  • An FXT file is being used with real spread enabled but real spread has failed to be enabled in the Birt's patch script.
• How come the backtest results differ from an optimization cycle results if same parameters are being used?