multiThreadedMeshProcess

Share your scripts with other members
lukeiamyourfather
Posts: 2880
Joined: Mon Oct 15, 2007 4:09 pm
Contact:

multiThreadedMeshProcess

Postby lukeiamyourfather » Fri Mar 28, 2008 11:12 pm

See my new script for multi threaded meshing in Linux:

http://www.nextlimit.com/nlscript/news_scripts.php?id=107

I appreciate feedback if you use the script. I hope you enjoy!


User avatar
tsn
Posts: 915
Joined: Fri Oct 27, 2006 10:36 am

multiThreadedMeshProcess

Postby tsn » Mon Mar 31, 2008 7:01 am

Thanks for sahring the script. It's a very interesting approach to start processes from an external Python script.

lukeiamyourfather
Posts: 2880
Joined: Mon Oct 15, 2007 4:09 pm
Contact:

multiThreadedMeshProcess

Postby lukeiamyourfather » Mon Mar 31, 2008 1:24 pm

tsn wrote: Thanks for sahring the script. It's a very interesting approach to start processes from an external Python script.

I wanted to use the built in threading class for Python, originally I started in RF but had problems getting it to work there. Running it outside of RF eliminated the problems I was having with the threading class.

It was kind of a mixed blessing though, because now it can be used to run on machines without opening RF. SSH into a few machines and run it to speed up the meshing process. Cheers!

lukeiamyourfather
Posts: 2880
Joined: Mon Oct 15, 2007 4:09 pm
Contact:

multiThreadedMeshProcess

Postby lukeiamyourfather » Fri Apr 18, 2008 9:23 pm

I updated the script on Next Limit's scripting site. It now includes support for Windows. Enjoy!

8cores
Posts: 169
Joined: Sun Nov 25, 2007 11:50 pm

multiThreadedMeshProcess

Postby 8cores » Fri Apr 18, 2008 10:14 pm

Hi luke,

could you tell me the difference between your script and this one (for windows users):

http://www.nextlimit.com/nlscript/ranking_scripts.php?id=95

created by:

www.negnog.de

for me is the same, but you are the master here and you could explain it better...

cheers

lukeiamyourfather
Posts: 2880
Joined: Mon Oct 15, 2007 4:09 pm
Contact:

multiThreadedMeshProcess

Postby lukeiamyourfather » Fri Apr 18, 2008 10:43 pm

8cores wrote: Hi luke,

could you tell me the difference between your script and this one (for windows users):

http://www.nextlimit.com/nlscript/ranking_scripts.php?id=95

created by:

www.negnog.de

for me is the same, but you are the master here and you could explain it better...

cheers

The script you linked to has a few problems with it that I don't like. It writes RFS and BAT files to the hard drive which requires write permissions and is sloppy in my opinion. It also ties it to Windows through the use of batch files instead of relying on built in Python functions to accomplish goals (keeping things portable and not tied to a particular OS). It's based on the original meshing script on the site which breaks meshing into frame ranges, this can lead to decreased efficiency if the meshes are larger and have more faces towards the end or beginning of a simulation because one set will finish very quickly and the other will take a very long time. The iteration step value can help combat this, which is a nice improvement over other scripts that use a similar method. RF must be running on the machine in GUI, which means meshing can't be started on remote machines as easily as a command line only approach.

It's an impressive script for sure, and shows proficiency with Python and batch but it's just not the way that I would do it, or did it. A queue and threads is much more efficient since all frames are put into a work queue and then threads grab the next unmeshed frame on the queue. This way if meshes are more complicated and have more faces in some parts of the simulation, the work will be distributed evenly and the meshing will finish much faster than if the meshes are broken into frame ranges for each thread.

Ideally the mesh generation functions would be multi threaded from the ground up so each thread would be sharing the work load for a single frame of mesh. That way there wouldn't be the need for 4, 8, 16, or however many times the memory since each thread is an independent instance of RF. Increased speed without the need for more memory than meshing with a single thread. Building meshes in the GUI would be faster too and tweaking meshes would be much faster. Hopefully this will be an improvement in future version of RF or at least in future tools like Pwrapper or similar. If not I may end up pursuing that on my own outside of RF because it's currently one of the weakest parts of the work flow. That's my two cents, nothing against the author of the other scripts but I just wanted to do things the way that I felt was best. And share it for others to use as well. Cheers!

8cores
Posts: 169
Joined: Sun Nov 25, 2007 11:50 pm

multiThreadedMeshProcess

Postby 8cores » Fri Apr 18, 2008 11:27 pm

as I said, you are the master here :first:, thanks for the explanation...


Return to “User Scripts”

Who is online

Users browsing this forum: No registered users and 1 guest