
In fact, when using ssh -X forwarding, make sure you do not manually set the DISPLAY variable. That's why on the remote system, the DISPLAY variable is "localhost" and not your client TCP/IP. By making a ssh -X connection you use the X Window server running on your computer to make it seem as it was running on your remote system. The idea of ssh X forwarding is to tunnel the X Window traffic. The server in this case is the X Window server, which always needs to run on the computer that is directly attached to the video controller and keyboard and mouse that you are using in front of you.

X Window is not the same as MS Windows or remote desktop and employs a client/server model. You certainly do not use ssh -X to start the X Window server or use it on the remote system.
