CSV Iterator and IF Statement bug

Automise Bugs
1

I am using Automise 5.0.0.1327, and I am iterating through a CSV file, and I am seeing two things that do not seem right.

The first line of the file is the header, and it contains the following:
ServerName,DatabaseName,Report Executed Date,Site_CD,CURRENCY CD,TOTAL ASSETS
Inside the iterator, I set the %aumServerName% variable equal to the first value in the line. This should be set to the word “ServerName”. However, when I set an IF statement to see if the %aumServerName% variable is not set to ServerName, it succeds because there is some weird character being put at the start of the variable.

So, I set the IF statement to check to see if the variable does not end with ServerName. The check fails (as it should) because the variable does actually end with ServerName. However, in the log, it shows the following:

image
image796×115 10.6 KB

As you can see, even though it shows the IF statement the way I created it, the Expanded Values shows a different logical check - “starts with” instead of “does not end with”.

The next line of file is then iterated, and it gets even weirder:

image
image732×129 5.83 KB

It somehow changed “does not end with” to "does not start with"

2

Are you able to send us the project (or a cut down version) and a csv file that shows this problem? That would make it much quicker to reproduce here (we will still try without it).

3

FWIW, I have found the issue with the changed text, it’s a copy paste error - will upload a fix shortly.

1 Like
4

I have reproduced the issue with the strange characters - it appears to be the encodiding BOM at the start of the file.

The library we are using (part of ms active scripting) does not appear to support files with a BOM :confounded: - looking for a solution to this now.

1 Like
5

What is a BOM? Is there something I can do on my end in the file to remove it?

6

A BOM is a byte order mark - used to indicate the file encoding. It is quite common in files these days.

Whether you can do anything about it depends on the data - if it’s pure ascii you can open it in notepad++ or another editor that allows you to change the encoding and then save it again (back it up first!).

I hope to have a new build today. We worked on two solutions to the issue yesterday, the first worked but relied on loading the entire file into memory - which could be a problem for large files - the second solution should work (reading the file line at a time) but we’re experiencing some compiler issues that we have to figure out.

7

I appreciate the work you are doing. In the meantime, the check I am using (whether the field value ends with ServerName) will work for me, and it should work the same way with the new Automise version.

8

Hi Jonathon

Here’s the new build with the fix for both issues

https://downloads.finalbuilder.com/downloads/automise/500/AT500_1358.exe
https://downloads.finalbuilder.com/downloads/automise/500/ATCMD500_1358.exe
https://downloads.finalbuilder.com/downloads/automise/500/AutomiseRunner500_1358.exe

This should deal with the file encoding correctly, I tested with utf-8 and utf-16 files (and ascii).

This version uses a buffered file stream so doesn’t load the entire file into memory - performance seems about the same as before.

1 Like