It would be really nice to be able to process Iterators asynchronously.
Hi Mike
This is something I have looked at before, it’s not a simple thing like it might be in a compiler. It will require creating a private variable name space for each iteration since most iterators work by setting variables (and that has other issues when the iterator is done, where does it take the variable values from…). I’m not saying it’s impossible, but I can’t make any promises other than to say I will add it to the todo list.
+1 vote from here also.
I can see the problems as you describe, but it would be very cool at some point if it was possible
Could you do something like:
Asynchronous Iterator uses a variable like %Servername%. When the project is ran, the iterator actually duplicates it’s child steps for every iteration and creates a variable of that name with GUID appended for uniqeness. That way you can actually see all of the steps run individually and asynchronously and log them.
So the child steps would have variables like
%Servername-979A1968-E428-11DE-844D-CC7255D89593%
%Servername-1C97E4E2-E429-11DE-AD05-C0B355D89593%
%Servernane-2023EF34-E429-11DE-8B54-D5B455D89593%
Casting my vote for this as well. It would be immensely useful for me.
I'll add my vote also.
For my case, I would need it with the XML Interator to be able to perform async actions based on some data retreived from a StoredProcedure.
I am also voting for this feature...
+1
Make it +10000
In the middle of evaluation, desinged this great XML, and what do I see ?
That when looping over instances/services everything is sequential...
This is major block.
Posted By Idan Mashaal on 20 Jun 2010 01:31 AM+1
Make it +10000In the middle of evaluation, desinged this great XML, and what do I see ?
That when looping over instances/services everything is sequential...
This is major block.
I'm not finding the part in our documentation that said it would work as you expect ? Which part of Iterator did you read as "run each iteration in parallel"? How many other build tools can run their loops in parallel? I'm finding it hard to believe that this is such a blocking issue.
What you are asking may happen one day, but it will take a lot of work and rearchitecting to make that happen. Iterators in FinalBuilder were never designed to be unrolled so it's not a simple thing to just conjure up.
I would also point out that in many cases we have helped customers improve the performance of their build process, and one of the common themes we see is overuse of threads. Running 50 I/O bound threads on a 4 core or even a 24 core machine is not going to make things any quicker than running 4 threads. So unless all those threads are working with completely separate resources (ie mix cpu bound with I/O bound) you quite often end up with worse performance because of the threading overhead (thread switching, locking etc). There are cases where unrolling loops might bring some performance improvements, but it's not the silver bullet that some people believe it to be.
while I agree with what you’re saying and I understand the concept of iteration is by its nature sequential, I still think it would be a great addition to FB, even in a changed/limited form. Not everyone is going to be doing any heavy lifting inside an async iterator. I am often performing actions on multiple servers via psexec or commands that let you specify a computer name as a parameter, and it would be great to kick these off in parallel.
What I’m getting at is I would find great use in being able to asynchronously call a set of actions with different parameters that come from a list. I make great use of asynchronous action groups where I can, but I haven’t been able to accomplish the above scenario asynchronously. One of the great things about FB is there’s usually another way to do something you want, but I haven’t found it yet.
Example:
I have 8 servers listed in a config file, load them into a variable, I loop through all 8 in an iterator and do IISReset. It would be nice to do this asynchronously and the async action would finish once all its 8 child actions finished, and return success/fail based on if any child action failed. If I knew in advance there were 8 servers, I could put 8 iisreset actions in an asnych group and be fine, but the amount of servers changes, so I can’t hard code things.
Hi Guys,
Please see the attached project for an example on how to achieve asynchronous iterating, it's not a perfect solution but a workaround none the less.
Cheers,
Paul.
I'll look into integrating this in my current solutions. As I read the sample project, it does make a lot of sense.
Thanks !
+ 1 on this feature.
If you look at the way .net 4.0 Tasks.Parrallel is implemented, they take the approach of optimising around the execution environment (.net handles the threading/performance issue on your behalf). I would hope that you might be able to leverage that codebase in Final Builder.
An alternate to Async lists, might be the ability to separate lists into x parts (say 4 to match the cores in your CPU) to help parallise a long running task with the existing Async Actions. Splitting a list into x parts and running the same actions via x lists running Async was the initial way I thought of working around this problem.
Example - list iteration performance problem.
I for instance have a set of SSRS report definitions (247 of them) that currently iterate through a list of files doing the following actions.
- read the RDL xml contents,
- change the xml,
- save the files back to RDLC file extensions.
It currently takes 38 minutes to achieve this on a quad core CPU via Final Builder Server.
My 2 cents - Jamie.
This is on my radar to investigate. BTW, the .net 4 task stuff won’t help us as 90% of FinalBuilder is unmanaged code.