Disk Performance Hands On, Part 3: RAID 5 Performance

<div><p><em>This is Part 3 in an ongoing series on disk performance. You can read the entire series by starting at the <a href="http://kendalvandyke.blogspot.com/2009/02/disk-performance-hands-on-series.html">Introduction</a>.</em></p> <p>RAID 5 is recommended for read intensive databases where you need to maximize space while retaining fault tolerance. Just like a RAID 10 drive achieving optimal performance depends on the right combination of stripe size, partition offset, and allocation unit size. For this set of tests I used the same hardware as my RAID 10 tests - Test Harness #1, a Dell PowerEdge 2950 (<a href="http://kendalvandyke.blogspot.com/2009/02/disk-performance-hands-on-part-1-test.html">view the test harness specifics here</a>).</p> <p><strong>Stripe Size & Allocation Unit Size</strong> <br />I’m following the same methodology and conditions as in <a href="http://kendalvandyke.blogspot.com/2009/02/disk-performance-hands-on-part-2-raid.html">part 2 of this series</a>. To recap, I used RAID stripe sizes of 4 KB, 8 KB, and 128 KB along with allocation unit sizes of 4 KB, 8 KB, and 64 KB on a partition with a default 32 KB offset. I focused on 8 KB random reads writes, which are the two types of operations SQL server performs on data files most of the time.</p> <p>Here’s what 8 KB random reads look like:</p> <p><img alt="IOs/sec, 8 KB random reads, 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2-Hlwgf0R1rHu7ARQFFJ6WEyJr6p6Ohc8472To4S3EOtzRwhgHAhcBlT3EeeBVgdVpIrS3cZdmsihZ4ovOBjUkjtYgCz2dx9ZY5qddkdW1n85hoT4MmMG7oJ0CMeRtYCkybH9n9x5JPkG/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random reads, 32 KB offset" width="483" /> <img alt="MBs/sec, 8 KB random reads, 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiduRWJA5N0c1sdsIYQ5zf8qI7hpke3uCzjfS9Mzo18NRBuFXdNu5M2f_7hYs2fvfULfJWNjtNQYP_nNcTOj4H1cHa1Ar6pQ4goYQ3i_C8AYhVTV8X1GBfNw1vddBAOWDdDID_v9Qur1y3y/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="MBs/sec, 8 KB random reads, 32 KB offset" width="483" /> <img alt="Avg Latency, 8 KB random reads, 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj32G334nG2NKoI2eJ-jRTGl4XC_gpu86BA2j-t2mu-_eO8zT_bsdIYfoe9X8yU_8ZLy_AGhcxQnfa8DXcBozpjxWCYhs0TqMCLn8Y7tMuaEy_04dlctQjqxeREjgA_HnM9W4ZVJBoR4e01/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="Avg Latency, 8 KB random reads, 32 KB offset" width="483" /> </p> <p>And here’s what 8 KB random writes look like:</p> <p><img alt="IOs/sec, 8 KB random writes, 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQjA4jMlguJZu98VtUqan7vlZFZ9SsigdrVH85XWIDiEWedWKZbRZCxZdRvuiZJ4ZDhszXnCye1goYjpPKX8tZjLV4i6HPD5grKRGwOiVArH2vNBIqjY4xQU1IEaxXNrEHVNWKHIDnPxpf/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random writes, 32 KB offset" width="483" /> <img alt="IOs/sec, 8 KB random writes, 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrdnJYTKTBmn7m6BgBt3BqTwA77BS60KPepJSGQBTNKday4SyideUS34J4A3HcOhFUE41yo26NpdMic_KoadZWk31H6KEbeQ-MwP3UdUAxMc3RUgKFg4JA5LsyrrxP7YkOacGiC89R9Q0F/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random writes, 32 KB offset" width="483" /> <img alt="Avg Latency, 8 KB random writes, 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjisHIa-eUu77qPXh4PKYyAhaELWUQj7tsKV6oTFQGVLjPjuFDwJP2MelGA06GZO3z2HOFsG62NrA_eB7par8njELCrLTI3OzIAu6vLN03RJBvnG-C92i5dv3ltV9YLaLUaP1dkKGN3lUlj/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="Avg Latency, 8 KB random writes, 32 KB offset" width="483" /> </p> <p>The results are in line with what we saw for RAID 10. A 128 KB stripe size offers the best overall performance. However, allocation unit size makes no difference when using a 32 KB partition offset. </p> <p><strong>Partition Offset</strong> <br />Now let’s look at how a 64 KB partition offset changes things.</p> <p>Here’s what 8 KB random reads look like:</p> <p><img alt="IOs/sec, 8 KB random reads, 64 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEin280N9wuC_GFZneHlY73CcXIbb0tkaTRNKui747fHZgUEpgXBcWY6mQC4Re7JlnPg_0qjF9OdrtoU0MzXyMs9PLHBmr5h2Aih9TJNEe_sQPPRLLuf82m8r_UqchSHSRJ0gpZ0DZzs6Z59/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random reads, 64 KB offset" width="483" /> <img alt="MBs/sec, 8 KB random reads, 64 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhccz3DMYoCvvvbvUv12dpKQ_y2zeb7cirpoqHqoJWbCEYxnjjltDCA7OV53y16NxIrmGB4dRwxhu50NQ_RyGBhrM8r7RZ9Tblv-MpvzEiQ6fGvFdNy8QRslUNy4qQzDptvCHf7SfyqFJPX/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="MBs/sec, 8 KB random reads, 64 KB offset" width="483" />  <img alt="Avg Latency, 8 KB random reads, 64 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXyij8KsRyat2vUUBYrCOkg-LDAWsNXhDtugmyOnPC2yGJEQ0WKiNbNqErdZAJJxeKDZL6zUXhhnNoIdTxJtiIBeByyvBpMN98zBsgi2n5TP-gEuEA8b_jEm6gCSjiNFYRJzoEht5isjyQ/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="Avg Latency, 8 KB random reads, 64 KB offset" width="483" /> </p> <p>And here’s what 8 KB random writes look like:</p> <p><img alt="IOs/sec, 8 KB random writes, 64 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2RqqcZ66qgZnHgcibdRO2O5MCfDm4Vpx97nNdqu_-HRZcic6w3g5LPwxXv0bY9D_CEqViwza2zY0lpz-Ia_RqLw2fdQQ5hrTq5GEQXbJNgYGfar4zqMArkI5XwEgqWcbUGkrF6nKwujcV/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random writes, 64 KB offset" width="483" /> <img alt="MBs/sec, 8 KB random writes, 64 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaTuwvZ_agEsCXUvubvvlsmKJ-pX2z-c4t2_m2WCDMfcIqcvITRCzKLVrspc1SPKnsJdB9wUXw73iBe9axqhBROh0JKTDRl8wPEAdMzznEbD-sE1Kntccxz16pfoEEFSv46PGuJw395N8U/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="MBs/sec, 8 KB random writes, 64 KB offset" width="483" /> <img alt="Avg Latency, 8 KB random writes, 64 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzgu3bygFxvbLYZwUW6Y_zWunsPY6IjE2hnmR8wNTGdx-RFO9VhoEOlx4tBXptq0zOzaRjmdkRzvHN7u_Bi1KhZWDxUB7HZYuPSyBHm6zjrJT3hBdReq1gSwb7skUYOanVNPRuoNcQN-TC/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="Avg Latency, 8 KB random writes, 64 KB offset" width="483" /> </p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p>Interestingly, with a 64 KB offset there’s very little difference between a 64 KB and 128 KB RAID stripe size for both read and write operations.</p> <p>What about how a 32 KB offset stacks up against a 64 KB offset? Since a 128 KB RAID stripe and a 64 KB allocation unit offered the best performance for a 32 KB offset, let’s go with the same for a 64 KB offset so that we can do an apples to apples comparison of the two.</p> <p>8 KB random reads:</p> <p><img alt="IOs/sec, 8 KB random reads, 64 KB vs 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwV-FI6Eii8nTTVnd5GAPVf62Rf7XqUQI99Tg2GCrNuA1eN3xukJgTCC20377zn71XrOuMYG1n_eqquW937JYPmdQ6YzMfxmkKEkyZ_qa2bV9wOKETp-YodC17lmDF-0hgsQZF3-t5mWft/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random reads, 64 KB vs 32 KB offset" width="483" /> <img alt="MBs/sec, 8 KB random reads, 64 KB vs 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNG7EUV6yO6t_o67OxYkc8KdTszJf1pagX7pbcj_IcWxl2EhDcyxhQ96ZNLDirF_5RCpqqToj12OyysX5rcN8vDtMqNoTt6AFXS3gEu7LKIAp6p4D5FM4h6oPydTtTaWx7AnPK1IYzmWva/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="MBs/sec, 8 KB random reads, 64 KB vs 32 KB offset" width="483" /> <img alt="Avg Latency, 8 KB random reads, 64 vs 32 KB KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwsYZ9AkAc7jtYdgfkmnwYD6YhErVDGPp5dhcvFI_35Pslj8KuBGjy5h5Wz8Ez0e05EQyxhXDmGCME69f4nPTjCAa5bBgexjUZl9QFv2BAtOm2bymKZ8cxWTMLAtuVolusAN96MGBKWT8J/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="Avg Latency, 8 KB random reads, 64 vs 32 KB KB offset" width="483" /> </p> <p>8 KB random writes:</p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p><img alt="IOs/sec, 8 KB random writes, 64 KB vs 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjTS_hksdP5_6tJyPUfbjBWtjRFskNbIuzZJxLX3iRU_OLJ0fnoM_1vWQP6k_6xnadr2U5Pv_JgbxVHLE1w_wXpbp2ELyWP7TnTRPQ20kLEcN-DkxjSgiHw0ME2xYp5uKlrriG2FwngAUJ/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="IOs/sec, 8 KB random writes, 64 KB vs 32 KB offset" width="483" /> <img alt="MBs/sec, 8 KB random writes, 64 KB vs 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgA5s8cqpvC7F7AdFB3S7gGv9rD8UGtV1Be8NiewXnhhq8D06ykIu0SC4kfT_8Waw8yzJGFZh-17JB_zHTKAzJ7M00kifCDjZUEgN_kyJkRKB5XRqaD9uNj0RlqoFDb_dx-0pw229WnA5j-/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="MBs/sec, 8 KB random writes, 64 KB vs 32 KB offset" width="483" /> <img alt="Avg Latency, 8 KB random writes, 64 KB vs 32 KB offset" border="0" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX7xSqxibhfQRcXwhSPAAY2JW6jyaz1z9Lx6FRlhaGW_MaKcBJRmqxixOWJXaWh6JNEES2qobKoUddybzxzgPRWaq4ajqTOx0bYKN5ca0cfKm0bTGIHWgHiIWrPDC8F8FPu8VDpMvi7sPQ/?imgmax=800" style="border-top-width: 0px; display: inline; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" title="Avg Latency, 8 KB random writes, 64 KB vs 32 KB offset" width="483" /></p> <p>Just as we saw with RAID 10, a 64 KB offset results in better performance than a 32 KB offset. Put into percentages, using a 64 KB offset results in a 5.7% improvement in IOs/sec and MBs/sec and an 8% improvement in average latency for 8 KB random reads. For 8 KB random writes using a 64 KB offset resulted in a 5.5% improvement in IOs/sec, MBs/sec, and average latency. </p> <p><strong>Conclusion <br /></strong>As I expected and the numbers showed, using a 64 KB offset results in the best performance. What I was surprised at, however, is how little of a difference allocation unit size made for 64 KB and 128 KB RAID stripes. We’re talking +\- 5 IOs\sec and MBs\sec. On paper that means for a 64 KB offset as long as you go with at least a 64 KB RAID stripe and a 4 KB allocation unit you’re going to be getting close to optimal performance. Unfortunately, although I had two other test harnesses I didn’t get the chance to see if this holds true when you throw more drives into the mix. My guess is with more drives the gap between numbers will become more apparent. Therefore I recommend you play it safe, go on the high end, and stick with a 128 KB RAID stripe, 64 KB offset, and 64 KB allocation unit size for your RAID 5 drives.</p> <p>Next, in <a href="http://kendalvandyke.blogspot.com/2009/02/disk-performance-hands-on-part-4-raid-1.html">Part 4</a> I will take a look at RAID 1 performance.</p> </div>

Share This

• Twitter
• Google+
• Linked In
• Facebook

Related Posts

Best Practices Performance SQL Server 2005 SQL Server 2008