However it looks longer and more complicated than it really is because you have been given lots of information in the question to lead you by fairly small and obvious steps to the solution and the diagram should help you see why you get a reflection in the line y=xtanθ.

You need to understand how a matrix represents a transformation of the plane and how it operates on each point to give its image under that transformation. See the problem 'The Matrix' from July 2003 and its solution for an explanation of how a transformation of the plane is given by a matrix and how you can find the image of a point by multiplying its vector by the matrix of the transformation.

Have a go, this is not so difficult.