Brender Render farm queue management system developement

I have written a few times on this blog about Brender. It is a little in-house system that I developed and used for the last 3 or 4 years to help me manage some rendering and scene management on different machines in our studio. To release the code to the public was always a wish to me, but not possible without some cleaning. It was first developed specifically for our network and machines, including a lot of hardcoded variables, machine names, IP’s and blender path.

Luckily recently I got the help of Francesco Siddi,  and we are working hard to have something downloadable (maybe before christmas?) .

A little work in progress screenshot

But be warned, this might not be the easiest render farm solution to set up and install  It is a system based on an apache web interface and mysql database. If you are interested to test it contact me (oenvoyage AT gmail) or Francesco (francesco.siddi AT gmail)

Indirect Lighting test

Yesterday i was playing around with the indirect lighting and environment lighting options in the renderer. Although the options are not that new anymore, and there have been many videos around, i never tried it before. So here is what i got after an afternoon, it can be tweaked much more but for a test it is enough

It is the same scene, same simulation (done in game engine then baked as f-curves, rescaled for timing). Scene has no light, only the emiting materials (emit 2.0) , and these are the scene world setting :

The factor level was boosted to 2, and a falloff effect added. The 2 first animations are the same renders, but with different color-corrections and glow values. As for the third one I slightly changed the balls, trying to use one material with a texture (with 2 values of colors) instead of 2 materials on different polygons (the middle glowy-emitting lines of polys and the rest of the ball).

You can download the .blend 3d scene here from blendswap.com: http://www.blendswap.com/blends/materials/glowing-balls-indirect-lightning/. It is creative commons license so you can of course use it as you want. I hope it can be helpful to some people.
The physics was done using the Game Engine physics, then recorded as f-curves using the Game->Record Animation option in the menus (when using Game Engine as render engine)

2.5 render times: sometimes faster, sometimes slower (+tips)

Recently i have been testing the renderings between both 2.5 and 2.49 and i have to say my feelings were mixed at the beginning. I have been opening some recent or older project files/images and just comparing the render times. Some of the scenes rendered faster in 2.5 (in fact most of them), but at the same time some were just slower! I have been testing with different settings and after a while i found out that the building of the raytree seems to take a lot on the some scenes that 2.5 renders slower.
So how can you improve that ?
Hum i was thinking to myself, strange that the raytree building takes lot of time. So i found the Raytrace Acceleration Structure in the performance tab in the Scene setting properties, and this is where everything happens. Basically the Auto setting work fine for most cases, except when there are a lot of instances/array of a same object and raytracing enabled like mirrors. In that case i found out the BLI BVH method works faster.
So whenever you have lots of faces and raytracing, don’t forget to try from Auto to some more adequate Acceleration Structure. I hope it can help someone in the future.

Some rendering doodling

Based on a texture of a carpet i found in some of my library hard drives, i wanted to see if a normal plain texture is enough to be faked using a kind of displacement or bump mapping, well these are 2 images of the same plane, just the displacement value was “slightly” too high in the case of the first image but i actually love that image much more than the second one.

and this is the kind of image, more like what i was trying to produce. I dont like the glass in the middle, but did not know what to put instead… maybe one day time and idea will help to put something more suitable…

Rendering on an 8-core with 2 blenders is faster

Today i am going to share some “trick” that i discovered on recent productions. First let me explain that I set up a small render farm environment (based on personal php/mysql system) in our studio.

brender renderfarming system

the basic
Our fastest machine is an 8-core mac. By checking the CPU usage during renders it clearly came out that there were times when the system was not using all cpus at max (either when transfering files, talking with server, getting the blend files, building shadow maps or other maybe non multi-threaded tasks). So the solution was to actually launch 2 instances of the render client script on the same machine and thus having 2 blenders rendering simulteanously . And now i can hear you …you are going to tell me “but hey it drops down your per-frame render time!!”:-) And yes sir you are perfectly correct! it slows down the per-frame rendering…but not enough down.
The numbers talk for themself :
One instance of blender rendering i get an average render time per frame : 38sec
With 2 instances the average per frame is not 1min16 as one would expect but 43sec.

 

A rapid useless but fun calculation tell us that for a 100 frames anim, i get 64mins of rendering (38sec*100frames/60) and with a “dual-blender” i get 36mins (80sec*50frames/60). Voila!
This scene (scene03 from current project) indeed includes quiet a good amount of compositing, which i do believe is not multithreaded yet.

Other tests : scene 04_office
2 blend 37 sec
1 blend 23 sec

And with the blender benchmark file from eofw.org _
2 blend average 31sec.
1 blend 17 secs. ( yes its slower from other mac pro results , but i renderer slightly bigger and OSA 8)

The .exr temp files problem
So we already used this “trick” on the H7 robot movie. But at that time we did not use full sample FSA. This is were we got the temp files problem. I do not really understand how blender works for saving the sample passes in an .exr temporary files, but it seemed that by launching the render client in a same user session, it got access both time on same destination. This made blender not able to either overwrite or simply write, and thus crashing or exiting without rendering.
I tried different solutions like having a separate blender installation for the second render client instance, as well as running the client script in different user in the terminal… strangely it did not affect anything.
The workaround solution we found (thanks to a patagonian yogi guru) was to start an other session on the same computer to run the render client.
I am suspecting my lame render farm settings to be the only mistake in that but at least in the end it worked.

So a tip to all people with a 8-core machine, or even 4-core i am sure u might get results, when rendering a long heavy animation: launch 2 instances of blender and use Touch Overwrite
for more infos on TouchOverwrite : http://www.blender.org/development/release-logs/blender-246/distributed-rendering-new-render-options/

please let me know if you do get some rendering speed improvement.

What to do during rendering…

Well this week and weekend was a bit of very hard work for Pablo and myself as we are in the last days of the new project, and it was rendering time! Render is often a very stressful time, because of render frames results, render farm not always working with all feature (i will talk about our little brender system later) and seeing that you missed to enabe an important layer at frame 327 or this kind of surprises…

but as you can see on our photos it can also be a kind of fun time Yes computers do work for us.
Pablo was kind of better at mini-minigolf than me.

But now its back to work and lets check those renders! I will post pictures as soon as i can.