News: A more realistic landscape is being developed! Table of contents affected by internet explorer issue. Best viewed with firefox. IE users can scroll downward.

Never been to the swiss? Wonder how lake geneva looks like? Here is your chance. Modem users may like the low resolution version. For screenshots and technical details, see below.

The gpu terragen plugin

The gpu project is rendering a interactive walktrough of lake geneva. Using the terragen virtual landscape renderer, and heightmaps of the NASA, we render the surroundings of lake geneva. he height-map is pretty accurate with a reasonable resolution. This means, that mountains should look 'real', especially from a distance.

For comparison, take a look of photographs and images that people took from Lake Geneva.

How to use

Simply walk trough the space, rotate the camera by clicking 'left' and 'right', or step forward. You can also click on the image itself to navigate, the image is divided in 6 regions, corresponding to the textual navigation links.
You may like to compare with original photographs.

Other worlds

LakeGeneva is not the only map, we are also rendering a walk on titan.

Technical information

• 13448 Images
• Image size 640x480, JPEG 50% compression, resulting in 20-50kB filesize per image.
• 503x503 terragen terrain size
• walk grid 41x41 spots
• each spots looks into 8 directions
• The 'zoom' of each spot is 90 degrees, so there will be 50% overlap.
• Simple PHP interface calculates the image index and shows the image.
• The terragen script was made using a tiny Python application, and used a heightmap for camera height positioning.

How does the rendering progress work

We make use of the gpu peer-to-peer distributed computing network. This is an experimental network that connects computers using a gnutella network.

18 GigaFlops

At time of writing, 30 march. 2005, a total of 15 computers are connected, delivering a total computing power of 20 GHz, Which rougly translates into 18 Gigaflops. Most of the nodes use AMD processors, the first 64-bit amd was detected which seems to easily compete 3GHz pentiums, although modern pentiums may have the advantage of hyperthreading, which s reported to have good results on terragen, almost doubling the speed when two concurrent jobs are run. Please note that terragen statistics are not accurate. So far, the gpu cluster has rendered over 6Gb on jpeg data! We made some movies before to test the distributed network.

Run jobs using the frontend

Here we run jobs using the 'terragen frontend' which manages the gpu plugin 'earthsim.dll'. earthsim (sorry for the slightly misplaced chosen name) launches a controlled terragen process, feeds it script, world, terrain and overlay files, and waits until terragen finishes. After that, the bitmap as outputted by terragen is converted to a 100% quality .jpg, which average sizes of about 250-300kB. Those jpeg images are uploaded to a central ftp server. This ftp server acts as 'project' server, and also contains source files (terrain, world, script etc) for terragen. New projects will be uploaded to the ftp server. GPU clients can fetch project files from this ftp server and upload the results. The frontend will broadcast job requests to clients. Clients can pick up a request and render 1 or more(time-limited) images for a particulair project. This also distributes multiple projects in an anarchisticst matter. Populair projects may be run with more frontends, thus have shorter request intervals.

The ftp locking mechanism

Here we see the ftp server's locking mechanism in action. This is a native windows server named "gemini" after it's dual processor setup, as seen from my workstation. Both run windows 2000 professional which has proven to be fast and reliable. Gemini runs the visual synapse FTP server on a high port to manage the projects.

Atomic ftp lock files

Some nerd stuff ;). The lock files are used to inform other clients that a particulair image is being processed, so they will not try to re-do. Also, the locking mechanism itself needs protection. Since requests are broadcasted, they are likely to arrive at clients at about the same time. In consequence, the ftp server will typically be accessed by more than one client at the same time, requesting typically the same lock file, since that is the next frame to be rendered.
To avoid double locking, we need to invent a atomic locking mechanism using ftp protocol. Simply uploading and verifying the file exists is not enough. timestamps aren't either. unique id in the lockfile an option. We found the solution by using the ftp rename command. Renames on a filesystem can be consided atomic: if the file already exists, an error is returned. So, if we upload a file 'lck_0001.tmp' and try to rename that file to 'lck_0001.lck', only one single client will be able to successfully rename this file, even if multiple clients uploaded it almost simultaniously.
Therefore the client that succeeded in renaming, can be absolutely sure it is the 'owner' of the lock file. The other client will just proceed and tries to lock the next available frame.
This locking method is supposed to work correctly on any ftp server, and is believed to be a sophisticated way to do atomic locking on ftp servers.

Batch convert

Here we use the fine imaging tool irfanview to batch convert the images, especially reducing the size. We use JPG 50% compression, which is a nice compromize for small (15-30Kb) image size (640x480) with reasonable quality. The files are copied over the network to the virtual domain lakegeneva.dubaron.com. This domain is run by the Visual Synapse HTTP server ;)

Monitoring the process

Of course we are entirely hooked on monitoring all nodes on our grid. GPU's netmapper shows us a list of what all members of our peer-to-peer grid are running.

Especially job 5867 attracks our attention, since that is a job id of a terragen project.

Notes on the image quality

First of all, let me state that i am not a terragen artist. Secondly, those images are rendered with the freeware version of terragen, that puts some limits on the quality.
Third, jpeg quality was reduced to obtain a reasonable file size for internet browsing (although modem users may not like it too much, broadband very recommended).
Fourth, keep in mind that there are 13448 images involved. Since this is one of our first walktrough experiments, we like to finish the job in reasonable time. A single image takes about 2-3 minutes to render on a modern PC, and another 20-30 seconds overhead for network transactions (reuesting lock, uploading data). Using gpu's distributed network, we hope to complete all images within about a week (5 permanent nodes rendering).
If this all is a success, we may re-do the rendering with improved terragen world and advantaged detail. Terragen artists are invited to create a more realistic terragen world map.

Since there are so many images involved, and browsing them is online, the quality of the compression counts but filesize also. There is no use in putting 100% quality jpeg files online, they would be needless big and your eyes can't tell the differce.
Paul has been so kind to create a test case, with various steps of compression, so you can view for yourself. The bitmap is the original as outputted by terragen. Using the free tool irfanview, and the commercial available jpg compressor 'xat' paul created a testcase. He created jpeg images, starting at 10% quality, increasing the quality with 10% steps until 100%. xat is used as comparison, since it claims to do adaptive jpeg encoding, scanning regions where more or less compression is needed. The results can be seen here. A zip file containing all images is also available.
Currently, the images on the website are compressed using irfanview, 50% quality, progressive encoding. rogressive encoding helps to display partially downloaded images in advanced, improving the walktrough experience at the price of a small extra file size.

Source - Do It Yourself

You want to play with this map? Think you can create a nicer world file for the next project? All files can be downloaded from here. Don't forget to download gpu to run your project distributed.

Costs

A 10-euro project by GPU - cost estimation
The total costs of this project are, human effort, licenses, hardware degradation and hosting not calculated.
Average time per frame: 139 seconds. Extra power consumed by the computer by not being idle: 40W. Number of frames: 13448
Total amount of image jpeg data: ~1.5Gb Price per kilowatthour: 20 eurocent.
Network overhead per image: 100Kb (mainly inefficient directory listing for large projects)
Price internet connection: unlimited bandwidth, so ( bandwidth used / total available bandwidth per month)*price per month Assuming server internet costs are typical, total clients the same.

Power: 13448 images * 139 seconds each = 519 hours * 40W = 20770W = 21kWh. At 20 cents/kwH makes about 4.15 ==> cpu power: 4.15
Network: 1.5Gb + 13448 * 100Kb = 1.5 + 1.3 = 2.8Gb. [note: overhead gpu network unknown, may be GB's as well] Max/month= 26Gb. (2.8Gb / 26Gb) * 25.00 = 2.70 ==> Bandwidth: 2.70
Disk space: (1.5Gb/160Gb)*100.00= 0.94 ; Optical medium for backup: 0.50 ==> Disk space: 1.44

Total costs:

power	4.15
server network	2.70
client network	2.70
Disk space	1.44
Total	10.99

As you can see, we exceeded the budget.

Project progress

The rendering started at march 27. 2005 15:15
Displayed is project progress per day, taking 23:59 as measurement point:

Date	# images completed	Progress (%)
March 27.	648	4,8%
March 28.	1977	14,7%
March 29.	3403	25,3%
March 30.	4683	34,8%
March 31.	5852	43,5%
April 1.	7289	54,2%
April 2.	8517	63,3%
April 3.	9260	68,9%
April 4.	9839	73,2%
April 5.	10833	80,6%
April 6.	11558	85,9%
April 7.	11631	86,5%
April 8.	12119	90,1%
April 9.	12241	91,0%
April 10.	12916	96,0%
April 11.	13448	100,0%

Project completed at April 11. 19:56:31 CET (Summertime, CEST) 2005

What made this project difficult?

Obviously, the large amount of data, and the distributed aspect. While the distributed network was already running, it was clear that as the project progressed, it took more and more time and bandwidth for the clients to search for new jobs to do and place the lockfiles. Fortunately, we did not encounter real issues, so the project could be successfully completed within reasonable time.
Another aspect is that of the viewpoint and varying terrain heights. A simple python script did the job with reasonable results. However, it became apparent that it is hard to create a terragen world that is realistic both from 'short' and 'long' distance. Also, the camera zoom was quite wide, whereas on 'normal' renderings the camera would zoom in a lot further. Right now we had 8 view directions from each spot, it would be nice to increase this to about 16 but that would also double the amount of images. Right now, the camera could view the world with a 90% angle, resulting in a 50% overlap when rotating, which seems a reasonable compromise.
It may be clear that a fancy feature, like varying camera heights, would also increase the number of images.
For future rendering projects, it would be nice to increase the amount of visible detail. This would increase rendering time, but not the amount of bandwidth and diskspace taken.
Concluding: with 13448 images involved, we hit the limit of the reasonable. Maybe the amount could be doubled or even more, but the amount of data grows proportionally.

Contact

You can contact us on the gpu chat, which provides a convenient place for idea exchange.
You can also post messages on the mailing list: http://lists.sourceforge.net/lists/listinfo/gpu-world

Credits

Brought to you by the GPU team:

• Tiziano aka Dangermaus (project leader)
• Rene aka Nanobit (plugin developer)
• Paul aka PaulAtreides
• Red
• Swiftstream
• Lwh
• johnatemps
• Krom
• Seeschloss
• many many others i forgot to mention