One way to permanently get rid of the Windows Firewall is to disable the service in the Services Manager, or set the startup type to “Manual”. Today I learned that it is not enough to disable the “Windows Firewall” service when you are on a Windows 7 or Windows 2008 R2 Server. You also have to disable the “Base Filtering Engine” if you do not want to hit a big red brick wall (also known as “Block mode”).

More on the topic in the Technet blog and this Technet article.