Compare and contrast the methods of solving systems of linear equations?

Question

Compare and contrast the methods of solving systems of linear equations

Answer 1

Well, what methods have you studied? Are you dealing with two equations in two unknowns or the more general cases.

For reasonable, dense, matrices, the LUP algorithm is generally preferred:
http://en.wikipedia.org/wiki/LU_decomposition

It is computes the results directly, with less effort and better numerical stability than trying to do Gaussian elimination directly.

However, there are situations where it is still better to use an iterative approach:
http://ikpe1101.ikp.kfa-juelich.de/briefbook_data_analysis/node156.html
http://stinet.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA141718

(When solving stiff ODE's the solution from one set of linear equations can often be used as the starting value for the next set, improving the efficiency of iteration dramatically)