{"id":817,"date":"2009-08-17T14:13:25","date_gmt":"2009-08-17T18:13:25","guid":{"rendered":"http:\/\/mat.tepper.cmu.edu\/blog\/?p=817"},"modified":"2009-08-17T14:13:25","modified_gmt":"2009-08-17T18:13:25","slug":"mittelmanns-benchmarks-cplex-verus-gurobi","status":"publish","type":"post","link":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/2009\/08\/17\/mittelmanns-benchmarks-cplex-verus-gurobi\/","title":{"rendered":"Mittelmann&#8217;s Benchmarks CPLEX verus Gurobi"},"content":{"rendered":"<p><a href=\"http:\/\/plato.asu.edu\/\">Hans Mittelmann<\/a> has some new benchmarks comparing CPLEX 12.1 with GUROBI 1.1.2 on various mixed integer linear programming instances (I last <a href=\"http:\/\/mat.tepper.cmu.edu\/blog\/?p=534\">wrote on these benchmarks last January<\/a> with earlier versions of both codes:\u00a0 be sure to check out the comments from that post since many of those comments apply to this also).\u00a0 He covers both <a href=\"http:\/\/plato.asu.edu\/ftp\/feas_bench.html\">feasibility<\/a> and <a href=\"http:\/\/plato.asu.edu\/ftp\/milpc.html\">optimization<\/a> problems.<\/p>\n<p>Checking the computation times for one set of optimizatin instances:<\/p>\n<pre>  problem     CPLEX1    GUROBI1     CPLEX4    GUROBI4\n------------------------------------------------------\nbc1            101         94         81         51\nbienst2        165         58         87         27\ndano3_5        148        618        276        362\nmark._4_0      225         45         86         36\nmark._5_0        f       6909          f       4368\nmkc1             5        113          5\nqap10           51         76         48         73\nseymour1       178        285        116        161\nswath2          13         10          5          6\nswath3          26        563         49         53\n30_05_100        7         11          4         11<\/pre>\n<p>The CPLEX1 and GUROBI1 use one processor;  the &#8230;4 versions use 4.\u00a0 The &#8220;f&#8221; means optimality was not proved in 7200 seconds (all times in seconds). \u00a0 The above table is not representative, so be sure to check out Hans&#8217; page.<\/p>\n<p>A couple conclusions that I come up with (your mileage may vary):<\/p>\n<ul>\n<li>Gurobi is a very impressive code:\u00a0 whether CPLEX or Gurobi will be faster on a particular instance is not predictable, which is an impressive feat.<\/li>\n<li>Parallel speedup is quite unpredictable.\u00a0 Often there is no, or minimal, speedup;\u00a0 sometimes there is huge speedup;\u00a0 sometimes there is a slowdown.\u00a0 None of this is surprising:\u00a0 the initial linear relaxation is hard to parallelize, and the nondeterministic nature of tree search can lead to getting &#8220;lucky&#8221; or &#8220;unlucky&#8221; in the parallel search.<\/li>\n<\/ul>\n<p>If we pull a few lines from the feasibility benchmarks:<\/p>\n<pre>============================================================\nproblem        CPLEX    FP-ab     FP-bfl    SCIP     GUROBI\n------------------------------------------------------------\natlanta-ip      848        23        21       391       189\ncore4872-1529     1       413        52       374         1\nds                1         -      5350        38         3\ngermanrr        183        10         6       200        19\nmomentum1         1        23        17      6257         2\nmomentum2      6181        47        71      1504      4118\nmomentum3         1       350       627         -         4\nmsc98-ip        185        19        25      1754        33\nneos16           47       236       241        43        72\nneos-506428    1338      2969      4531      1739      2694<\/pre>\n<p>(FP are Feasibilty Pump approaches; <a href=\"http:\/\/scip.zib.de\/\">SCIP<\/a> is a fast non-commercial code). \u00a0 Here the results are all over the map:\u00a0 practically anything can win and anything can lose and there can be\u00a0 three orders of magnitude difference between running times.<\/p>\n<p>Now speed is not the only thing to look for in an optimization code (support, interfaces, price are a few other things to look at), but these benchmarks are very useful in trying to compare codes.\u00a0 Thanks Hans!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hans Mittelmann has some new benchmarks comparing CPLEX 12.1 with GUROBI 1.1.2 on various mixed integer linear programming instances (I last wrote on these benchmarks last January with earlier versions of both codes:\u00a0 be sure to check out the comments from that post since many of those comments apply to this also).\u00a0 He covers both &hellip; <a href=\"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/2009\/08\/17\/mittelmanns-benchmarks-cplex-verus-gurobi\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Mittelmann&#8217;s Benchmarks CPLEX verus Gurobi&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,12],"tags":[],"class_list":["post-817","post","type-post","status-publish","format-standard","hentry","category-companies","category-computing"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/817","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=817"}],"version-history":[{"count":0,"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/817\/revisions"}],"wp:attachment":[{"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=817"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=817"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mat.tepper.cmu.edu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=817"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}