A closer look at ASM rebalance, Part II: Disks have been dropped

This article is the second Part of the “A closer look at ASM rebalance” series:

  1. Part I: Disks have been added.
  2. Part II: Disks have been dropped.
  3. Part III: Disks have been added and dropped (at the same time).

If you are not familiar with ASM rebalance I would suggest first to read those 2 blog posts written by Bane Radulovic:

In this part II I want to visualize the rebalance operation (with 3 power values: 2,6 and 11) after disks have been dropped.

To do so, on a 2 nodes Extended Rac Cluster (11.2.0.4), I dropped 2 disks into the DATA diskgroup (created with an ASM Allocation Unit of 4MB) and launched (connected on +ASM1):

  1. alter diskgroup DATA rebalance power 2; (At 09:09 AM).
  2. alter diskgroup DATA rebalance power 6; (At 09:19 AM).
  3. alter diskgroup DATA rebalance power 11; (At 09:29 AM).

And then I waited until it finished (means v$asm_operation returns no rows for the DATA diskgroup).

Note that 2) and 3) interrupted the rebalance in progress and launched a new one with a new power.

During this amount of time I collected the ASM performance metrics that way for the DATA diskgroup only.

I’ll present the results with Tableau (For each Graph I’ll keep the “columns”, “rows” and “marks” shelf into the print screen so that you can reproduce).

Note: There is no database activity on the Host where the rebalance has been launched.

Here are the results:

First let’s verify that the whole rebalance activity has been done on the +ASM1 instance (As I launched the rebalance operations from it).

Screen Shot 2014-08-31 at 18.50.48

We can see:

  1. That all Read and Write rebalance activity has been done on +ASM1 .
  2. That the read throughput is very close to the write throughput on +ASM1.
  3. The impact of the power values (2,6 and 11) on the throughput.

Now I would like to compare the behavior of 2 Sets of Disks: The disks that have been dropped and the disks that will remain into the DATA diskgroup.

To do so, let’s create in Tableau a SET that contains the 2 dropped disks.

Screen Shot 2014-08-20 at 21.27.34

Let’s call it “Dropped Disks”

Screen Shot 2014-08-31 at 18.53.04

So that now we are able to display the ASM metrics IN this set (the 2 dropped disks) and OUT this set (the disks that will remain into the DATA diskgroup).

I will filter the metrics on ASM1 only (to avoid any “little parasites” coming from ASM2).

Let’s visualize the Reads/s and Writes/s metrics:

Screen Shot 2014-08-31 at 18.57.08

We can see that during the 3 rebalances:

  1. No writes on the dropped disks.
  2. Number of Reads/s increasing on the dropped disks depending of the power values.
  3. Reads/s and Writes/s both increasing on the remaining disks depending of the power values.
  • Are 1, 2 and 3 surprising? No.

Let’s visualize the Kby Read/s and Kby Write/s metrics:

Screen Shot 2014-08-31 at 19.00.33

We can see that during the 3 rebalances:

  1. No Kby Write/s on the dropped disks.
  2. Number of Kby Read/s increasing on the dropped disks depending of the power values.
  3. Kby Read/s and Kby Write/s both increasing on the remaining disks depending of the power values.

Are 1, 2 and 3 surprising? No.

Let’s visualize the Average By/Read and Average By/Write metrics:

Important remark regarding the averages computation/display: The By/Read and By/Write measures depend on the number of reads. So the averages have to be calculated using Weighted Averages.

Let’s create the calculated field in Tableau for the By/Read Weighted Average:

Screen Shot 2014-08-20 at 21.56.49

The same has to be done for the By/Write Weighted Average.

Let’s see the result:

Screen Shot 2014-09-01 at 21.33.48

We can see:

  1. The Avg By/Read on the dropped disks is about the same (about 1MB) whatever the power value is.
  2. The Avg By/Read and Avg By/Write on the remaining disks is about the same (about 1MB) whatever the power value is.
  • Are 1 and 2 surprising? No for the behaviour, Yes (at least for me) for the 1MB value as the ASM allocation unit is 4MB.

Now that we have seen all those metrics, we can ask:

Q1: The ASM Allocation Unit size is 4MB and the Avg By/Read is stucked to 1MB,why?

I don’t have the answer yet, it will be the subject of another post.

Two remarks before to conclude:

  1. The ASM rebalance activity is not recorded into the v$asm_disk_iostat viewIt is recorded into the v$asm_disk_stat view. So, if you are using the asm_metrics utility, you have to change the asm_feature_version variable to a value > your ASM instance version.
  2. I tested with compatible.asm set to 10.1 and 11.2.0.2 and observed the same behaviour for all those metrics.

Conclusion of Part II:

  • Nothing surprising except (at least for me) that the Avg By/Read is stucked to 1MB (While the allocation unit is 4MB).
  • I’ll update this post with ASM 12c results as soon as I can (if something new needs to be told).
Advertisements
This entry was posted in ASM, Tableau. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s