I am sure, that if you work in the architectural visualization business you have struggled with the “hedges” problem more than once.
The “natural” way to do it would be to add the hedges directly in 3d. However, this method may make the scene to heavy, and it is likely that you will encounter memory errors
If you use 3d max and vray you can work it out by converting your 3d hedges into vray proxies; but still that will have an impact on the rendering time.
Depending on the angle of your rendering you may be able to use opacity maps and you will obtain decent results.
But what to do when your camera is not perpendicular to he opacity maps? (Everybody knows that in this case, the opacity maps will give you really unrealistic results due to the fact that it would be visible that there is no volume there)
After having struggled myself with this problem for some time, I have come to the conclusion that the following workflow will give the best results.
1) Render your scene with just a simple geometry instead of the complex 3d hedge model (after having applied a hedge texture that will generate the appropriate GI color)
2) Find a good highres picture of a hedge (or take one yourself) and crop out the background in photoshop. This is the hardest part because you have to crop it very carefully along each leaf in order to obtain a good result
3) Drag the layer of the resulting hedge over the base rendering and skew it until it matches the perspective of the base volume.
4) Now drag another copy of the layer and skew it so it will match the other side of the hedge. Crop the unnecessary part, following the contours of each leaf (like you did when you initially when you cropped the hedge from the original photo)
(you may want to hide the other layer, since you don’t need it now) After having done that adjust the brightness and contrast to make the new layer a little darker (or lighter, depending on the case, but the main idea is to suggest the volume)
5)It is starting to look ok but until now it seems a little “blocky”; so let’s do a little fine tuning. Duplicate the old layer and make it darker (similar to the new face of the hedge). You should now have the dark layer placed above the original one. Using a smooth and wide brush start deleting the layer above from right to left (in our case) until you are getting closer to the other face.. You should obtain a smoother result
6) You can now fine tune it even further (though you can skip this step if your hedge is placed at medium or far distance from the camera). Merge the layers that you have so far and again make another copy of the resulting one. Make the one below darker and start deleting the base of the one above in a similar manner like you did in the previous step. This way your hedge will be a little darker at the bottom and lighter in the upper part.
7)In the next step, crop the part where it intersects with other elements (in this case, the car) and simply press delete
8)It is now time to take care of the shadows that are projected on the hedge. A basic way to do this would be to simply select the area where the shadow is and lower the brightness value. However, I prefer to use a similar method to the one explained above (with 2 layers – a dark one below, and the original above). I find it easier to fine tune it this way.
9)Ok, you are almost there! This is actually the final step, and again depending on the angle and on how far is your hedge placed from the camera you may skip it. In this last step you should correct the shadow projected by the hedge; the initial one that came directly from the render, is a straight line and it is not correct. The easiest way to “brake” it either by using a clone tool with a leaf brush or something similar
10) No more steps! You are done
Even if this may seem a little ambiguous at a first glace, once you get the hang of it becomes really easy and may be a real life saver (I know for me it has been more than once).
The shorter version of the tutorial would be:
-Render the scene with a 3d textured box instead of a 3d hedge model or an opacity map
-Crop a photo of a hedge in photoshop and paste it over the original rendering.
-Skew it and crop it so it matches the perspective of all the face of the 3d textured box in the rendering
-Adjust the shadows projected and received by the 3d box and you are done.
If you have any questions don’t hesitate to ask!