Bald and Pantless

This year we went on a family vacation to Black Bear Lodge in WI. Northern Wisconsin is beautiful. Perhaps it’s being surrounded by the wood that allows you to relax and forget the everyday. It was a wonderful sunny day, sure it was warm but all of us were soaking up the sun and really enjoying ourselves. At the lodge they have a small beach area where the kids swim and play. Most of the kids enjoyed splashing in the water, building sand castles and swimming out to the raft.

I’m enjoying sitting by the beach watching the girls, of which I have three, playing. Drinking a beer I feel a tap on my head and the baiting laughter of Jude calling me, “baldy!” Have you ever been so relaxed that despite the taunts you choose not chase the little fiend? Yea, I was in that wonderful place. Still I’m in my early forties and I run 3 miles a day so in all fairness it was only right that I give some warning, “I’ll get you and
through you in the water!” Jude wasn’t wearing a swim suit so the warning carrying a bit of weight. The heckles changed from ‘baldy’ too, “nice halo” or some other witty remark. My wife Carol sat down was having a laugh as Jude was really working the entire beach with his colorful taunts. Very slowly I set my beer down waiting for Jude to run by and tap my well-earned bald spot. Now I would like to say that I leaped up like a cheetah after it pry but, that simply wasn’t the truth. Instead Jude had a bit of a head start. [1] Thanks to an error in Jude’s judgment, he ran on the pier to escape me. This is going to be sweet! If I get pulled in the water, who cares! I was wearing a swimming suit and the cold refreshing water would feel good. I slowed my approach. Yes, let’s make him sweat it out, “I got you now!”

You’ll never guess what happens next? I feel someone grab the back of my swim suit and start pulling down. This is when the world slows down, everything kind of stops and you realize that your butt is on display. Yes that’s right kids, adults, fish and even the eagle high above saw smiling at my demise. Who could be defending Jude? The entire beach knew why I was going to throw him in the water. No one yelled, “Stop he’s only a kid!” They all understood and were all too happy to watch him go swimming! Carefully, I spun around while holding the front of my swim suit. [2] I’ll be honest. I was looking
straight into the eyes of an invisible person. “Daddy, you no throw Jude in the water,” said Sophia in her motherly voice while shaking a finger at me for good measure. Sophia is my youngest, about 4 years old and apparently fond of Jude. That’s it right? Oh you know what I mean. You can’t be made and so you laugh. We did agree that it was okay to throw Jude in the water a little bit. So Jude got wet up to his knees.

The rest of vacation I watched Sophia as she gave Jude and  Cian Gibson her attention. It was a wonderful vacation that full of some great  laughs. I’ll save those for another blog.

[1] Hmm.. I did mention I was in my forties, right?

[2] Everyone got a peek at the mountains, no reason to show them the forest!

Posted in Uncategorized | Leave a comment

Ahh…

I was working from home and walked into the kitchen. I wasn’t trying to be quiet, Dads usually aren’t, but my oldest daughter didn’t notice me. I love her hair as it’s long, blond and naturally curly. The curly part she got from Dad (but don’t tell her). So I stopped short of walking into the kitchen as I saw her drinking the last of the peach juice. After the last drop from the jar was gone she let out a small satisfying, “Ahh…” and put the jar down. Wow, I totally forgot! Remember having those little assorted fruit cups for lunch and the best part of it was the juice. How about pickle juice? I loved to drink pickle juice just as much as I enjoyed eating pickles. It’s a shame that as we get older we forget just how good that peach juice was, right? Some of those other “Ahh…” moments include the high school years when running cross country or track on a brutally hot day. We didn’t stop running until we reached the water fountains in the school hallways. Who can forget taking a bike ride with the gang out to 7-11, buying a Slurpee and getting the brain freeze on. How about the opposite: playing outside in the snow all day and coming in to some hot chocolate. What are some of those liquid “Ahh…” moments that you remember?

Our kids have a lot more than we did at their age but most of them don’t know what its like to drink from a cold bottle of Coke. Looking back I have to say the most refreshing drinks I’ve ever had are, “Water, peach juice, hot chocolate and pickle juice.” The simplest things in life can be the most gratifying.

Posted in Uncategorized | Tagged | 1 Comment

RAID 5 Part 2

Continuation from: Designing a RAID 5 group

So to continue our discussion on RAID 5 lets investigate how EMC arrays use cache to minimize the impact of the read-read-write-write penalty. First let’s look at the events without having an array cache. When changing a block of information on a RAID 5 group the following events occur:

RAID 5 Steps without an Array Cache

  1. Write of new data pending
  2. Reads the unmodified data (B1)
  3. Reads the unmodified parity (Bp)
  4. Recalculates the new parity
    1. Requires new data (Step 1) & read of unmodified data (Step 2)
    2. New parity generated
  5. Writes the modified data (B1)
  6. Writes the new parity (Bp)
  7. Acknowledge write to database

I have included a picture to help illustrate the process:

RAID 5: Generic Steps

RAID 5:Generic Steps

Notice that the write is not acknowledged to the database until step 7. Unfortunately, this RAID 5 write penalty will impact performance of an Oracle Database or other applications unless there is a means to off load this penalty. In contrast, let’s look at how the cache in EMC arrays significantly improves the performance of the RAID 5 write penalty. Using the array cache the events change too:

RAID 5 Steps with an Array Cache

  1. Acknowledge write to database
  2. Read the unmodified data (B1) into cache
    (Can happen in parallel with step 4 for better performance)
  3. Write modified data to disk (B1)
    (Asynchronous operation)
  4. Read unmodified parity into cache (Bp)
  5. Recalculate the new parity
  6. Write the new parity to disk (Bp)
    (Asynchronous operation)

I have included a picture to show the process with an EMC array cache:

RAID 5: Using EMC Array Cache

RAID 5: Using EMC Array Cache

As the first step in the process is immediately acknowledging the write back to the database the array cache effectively removed the write penalty by off loading the work. Other efficiencies include:

  • The data write in Step 3 and parity write in Step 6 can happen asynchronously / independently which improves performance.
  • The ability to read the  unmodified data (Step 1) in parallel with reading the unmodified parity improves performance as two reads are consolidated into one read event
  • Both the new data and parity remain in cache for better performance

As you might expect an important consideration is ensuring the array cache is sized to support the database(s) and other applications. Might be a good question for your storage administrator or EMC consultant.

We have shown how one random write on a RAID 5 array works every efficiently using the array cache. How about using RAID 5 for supporting batch process which involves large sequential write operations?

RAID 5: Sequential Writes

RAID 5: Sequential Writes

Using the image above we see that the array cache is able to queue several recalculations of parity and de-stage them to disk in one write operation. So with large sequential writes using the array cache we have the benefit of immediate acknowledgment of the write requests back to the database and the efficiencies gained by reducing the number of writes back to the disk. I hope this provides some insights into how EMC works to provide your business performance benefits in using RAID 5 groups.

Thanks too: Kevin Foote, Sanjay Narahari, Matt Kaberlein and Yaron Dar for proofing and providing guidance on this post!

Posted in Uncategorized | 1 Comment

Remembering Sunny Brook Farm

I’m fifteen years old and we’re striking out on another family vacation to some place called, “Sunny Brook Farm”. Now all bets are off as I’ve never been to this place but a vacation is a vacation, right? What’s cool about our station wagon is that we can store most of our luggage in the carrier on the roof. That leaves the back of the station wagon for us to stretch out and relax. Traveling was fun as we played games like “I spy” and “alphabet” or just telling jokes. Maybe even some signing over the loud radio. We were traveling with two other families so it was fun convincing some of the younger kids that we were being followed by Martians! I saw the Martian space ship all the time but they just missed catching it. I guess it takes an experienced observer!

We could only drink a bottle of coke per hour, Mom’s rule. So the trip took about two and half cokes until we finally pulled into Sunny Brook Farm. Hmm.. there’s a big white kinda farmer style house in front which we walked into to check in. Our parents gave us our cabin keys with the orders to, “Open the doors and come back to help carry the stuff in.” Kids and keys, you know it turned into a race. I’m just over 6 feet tall and about 125 pounds. Why so skinny you ask? I’ve been running as long as I can remember that includes sports like track and cross country. Anyway I waited at our cabin for my brother as he was the lucky one that got the key. Not much to say about the cabins as they were knotty pine planks on the outside and small on the inside. Still the refrigerator was large enough for the coke bottles and some snacks, “Good enough, right?” Don’t really plan on sleeping so didn’t bother looking at the beds.

When we were carrying the luggage we were able to quickly pickup on the hot spots. It seems that the cabins were arranged in a big rectangle and in the middle was a pool, yep going to hit that fast, a volleyball court and some grassy area. There were picnic tables near the cabins but we weren’t interested in eating. Just as fast we could rip our swim trunks out we were swimming. It was a warm sunny day and the pool was wonderful especially after the road trip. At first it was just swimming but then it turned into races followed by marco-polo. Other folks were arriving and soon the pool was filling up with other kids. That was just fine as we made friends quickly and the games became more fun. Then she happened. I don’t know her name but she was beautiful. She was about my age had really blond hair that seemed to catch the sunlight and hold it. She had fair skin with some freckles and these amazing blue eyes that make the pool water look dull by comparison. I’m a little on the shy side but I’ll climb any mountain to meet her.

Time seemed to whip by as the dinner bell was ringing and some teenager was shouting, “Dinner is now being served!” The food was good but not having to clean up afterwards was even better. We couldn’t go swimming as we just ate, Mom’s rules again. So most of the kids hung out at the volleyball court and soon afterwards some good games started. Me and some of the bigger guys decided to be on one team so that we could deliver a whopping to the smaller kids. Dang-it I still don’t know her name but she was wearing a t-shirt and some cutoff jean shorts. Okay I’m a teenager I’ll get over the embarrassment later so trust me when I say, “she is HOT!” As luck would have it she was playing on the other side. No opportunities to meet her, at least not yet. We played until it was too dark to see the ball and then we played a little more.

Some older and very cool teens stopped by asking, “You guys want to go to the teen room” This place was awesome as there were names all over the inside. I mean, if you were there you made sure your name was somewhere in the room. Some kids spray painted on the walls, some used markers on the ceiling and other engraved their names anywhere anyhow. This was just one room with one light and not much place to even sit. More has to be said about the lighting as the one bulb was barely enough to light every corner of the small room. Anyway, the older teens set about making us tell our names and where we were from. JILL! Yep, her name was Jill. Next one of the teen leaders said, “We’re going to play a game called, “Follow the Train. It goes like this I’m (the guy’s name was Tom) going to hold Suzie by hip and we are going to walk around the room.” Oh yea, I forgot to tell you that we formed a big square as all the kids were leaning against the walls while introducing ourselves. “As we walk around the room I’m going to pick someone to follow us outside. NO body but those on the train can go or peek outside! When we come back the last person on the train gets the pick the next person.”

We all nodded and whispered to each other things like, “What the hell is this game about?” Suzie turned on the boom box and the game started. Suzie started by the door walking around the room and Tom picked out another girl. The ‘Train’ went out the door which left us all listening to music. It took a few minutes but the Train came back through the door and started slowly walking around the room. The girl at the end of the train passed me up and picked some other guy. This time when they went outside it took even longer. We were all talking and none of us had the courage enough to peek outside. Finally, the Train came into the room and just like the other times started making its way around all the teens. The guy at the end of the Train picked Jill. Well as you have guessed they went outside and took even longer. This great new song on the radio called, “Sunglasses At Night” just started as the Train came through the door.

Slowly they walked around and you’ll never guess what just happened. Jill picked me! My gut was tied in knots, my mind was racing and my hands were on her hips! I followed Jill out the door and the Train stopped a short ways from the room. Suzie and Tom locked lips, like forever. Then Tom turned and kissed the girl he had picked. “Holy crap! I’m going to kiss Jill!” Tom didn’t kiss the girl nearly as long but, he kissed her and if I’m guessing right she is going to kiss the next guy. “YESSSS! She did, She did, She turned and kissed the next guy.” He turned and kissed Jill. “Can you believe this? The hottest girl at Sunny Brook Farm is going to kiss me!” Time seemed to stand still, my breaths were shallow in anticipation, you could see the stars and the air carried the sweet cool smell of Jill. Then she turned to me, even at night those blue eyes were brilliant. Jill took a small step towards me while looking in my eyes and …

She SLAPPED me!

I was stunned, no words to described it. She was still looking in my eyes and wearing a small smile as some of the other kids were laughing. “Sam! Okay man hears how it works” said Tom from somewhere in the dark, dark world. “It’s your turn to pick some girl out and when she thinks you are going to kiss her then you slap her. Got it man?” “Wait a moment” I thought to myself, “This isn’t that bad!” Tom was looking at me to make sure I got it. “Yea, no problem,” as I kinda laughed it off. The Train was moving again and there was a room filled of people wondering what the hell was going on. I picked a girl out but she wasn’t nearly as cute a Jill. Outside again and the kissing started. The same rush of anticipation seemed to mount as the kissing progressed closer and closer to Jill. She turned, moved closer and gave me the most wonderful kiss. The next kiss was longer, the next longer still until finally someone yelled, “Busted.”

That night was magic. Summer love is so brief. To Jill: I may have gotten your name wrong but I remember your face as if it was yesterday. You rocked my world. Thanks for picking me!

Sometimes it’s interesting how you remember stories. For me the thought chain worked like this: Movie “The Diary of a Wimpy Kid” (funny) > Song in the movie by Bonnie Tyler “Total Eclipse of the Heart”  > Lyrics “I get a little bit nervous that the best of all the years have gone by” > Sunny Brook Farm!

Seems like a good story for a Friday! To those folks who shared the Sunny Brook Farm experience: Please provide some feedback and I’ll update the story! Lastly, thanks to my wonderful wife Carol for proof reading the story. I love you honey bunny!

Posted in Uncategorized | Tagged | 3 Comments

Orange Day

I had gone to be early last evening. My body was tired and my brain was fried. Thankfully, I gave the girls their bath and they were nicely tucked away in their beds too. It was one of those nights that you sleep so deep not the thunder or pounding rain could wake you up. Yea it was great. I woke up early just before sun raise so I made coffee and sat on the front porch. Then it happened. The orange colors of the sun raising seemed to be trapped by the low hanging clouds. The world was aglow in orange. Amazing.

Posted in Uncategorized | Tagged | Leave a comment

Designing a RAID 5 group

In a blog called, “Disk I/O for Oracle DBAs” we talked about a simple way of estimating disk I/O.  Below is a list of disk types and relative IOPS:

Enterprise Flash Drives (EFD): between 2000 – 4000 IOPS
Fibre Channel 15k RPM drives: 180 IOPS
Fibre Channel 10k RPM drives: 120 IOPS
SATA 7200k RPM drives: 90 IOPS

Then we used some simple math to estimate IOPS:

Number of disks = 50
Type of disk = Fibre Channel 15k RPM drives: 180 IOPS

So the total IOPS would look like: 50 disks * 180 IOPS = ~9000 IOPS

Let’s complicate the picture a bit by adding RAID 5 into the mix. As most DBAs know RAID 5 uses block-level striping with parity data to evenly distribute both data and parity across all the disks in the RAID 5 group. Use this link for researching RAID 5 and other levels of RAID:

Wikipedia The Free Encyclopedia: Standard RAID Levels
http://en.wikipedia.org/wiki/Standard_RAID_levels

By the way what does RAID stand for? Redundant Array of Independent or Inexpensive Disks. Now this is where we begin to drive into some complexity as different models of storage arrays have guidelines for creating RAID 5 groups. For example, the recommendation for a RAID 5 group on one storage array might be 4 + 1 which means for four (4) disks in terms of capacity and one used for parity. The calculation for determining capacity looks like this:

Smallest disk capacity  x (total number of disk  – 1) or the short version: Smin x (n – 1)

Let’s say we are using 1 TB disks with a 4 + 1 RAID disk group then our calculation looks something like this:

1 TB x (5 – 1)  = 4 TBs of total data capacity!

Great we know the capacity and it’s simple to estimate our disk IOPS:

4 Fibre Channel 15k RPM drives x 180 IOPS = ~720 IOPS

So in summary with a RAID 5 (4 + 1) group made of 1 TB 15k RPM drives we can estimate:

Group Size                 Estimated Capacity               Estimated IOPS
(4 + 1)                         4 TB                                        ~720

Did you see what happened? We lost both capacity and IOPS in using RAID 5! Due to the parity stripping the equivalent of one disk is dedicated to providing some resiliency in the case of a disk failure. Most of us would gladly dedicate one disk if it meant keeping our Oracle databases up and running. Let’s do the same calculations for a larger 6 + 1 disk group keeping everything else the same.

Capacity:  1 TB x (7 – 1) = 6 TB
IOPS:        6 x 180 IOPS = 1080 IOPS

Lets add to our table:

Group Size                 Estimated Capacity               Estimated IOPS
(4 + 1)                         4 TB                                        ~720
(6 + 1)                         6 TB                                        ~1080

Interesting, as a RAID 5 group consisting of 7 disks offers 33% more IOPS than a group with 5 disks and with the same resiliency for each group. It would seem the larger the RAID group the less disk capacity lost and the greater the performance. Remember that the trump card is the RAID 5 design that works most efficiently in your EMC storage array.

Now lets have some fun by adding a RAID 5 (4+1) Enterprise Flash Drive (EFD) into our table.

Capacity:  400 MB x (5 – 1) = 1.6 TB
IOPS:        4 x 2000 IOPS = 8000 IOPS

Group Size                 Estimated Capacity               Estimated IOPS
(4 + 1)                         4 TB                                        ~720
(6 + 1)                         6 TB                                        ~1080
(4 +1)                          1.6 TB                                     ~8000

The downside is we lose some capacity in that our EFD disks are only 400 GB. Still the ~8000 IOPS offer us the opportunity to significantly reduce access times and increase the performance of our database. One idea is to use the new 11gR2 Flash Cache feature with a small EFD RAID 5 group to increase the overall performance of the database.

In the next blog we will talk about how EMC improves the performance of RAID 5 using our array caching. Are you interested in some assistance with storage design contact your EMC Sales Representative or use this link:

http://www.emc.com/services/portfolio/implementation/design-implementation/index.htm

Have a great day

Posted in Uncategorized | Tagged | 2 Comments

Is Oracle Chipping Away at Their Greatest Strength?

So you just purchased a new house and it’s beautiful. The floor plan matches what you had in mind, your friends will be impressed on how well you decorated and you can finally relax with some personal space. Working with the builder was a good experience in that the company had all the right expertise and talent to build everything correctly. In fact, sitting in a chair you can control everything in the house from the lights, cooling, heating and everything else you might want. Only one thing left to do is buy new kitchen appliances. You’ve been doing research and there is a new generation of appliances that will cook faster, cool better and make your life easier.

It’s a rainy Saturday morning which makes the day perfect to do you’re appliance shopping. Because you got the jump on the day the store is relatively empty and you have a sales person waiting to talk to someone like you. This is going to be easy as you know what you want and you have already checked the store inventory online to make sure there are will be no delivery problems. In talking with the sales person you tell him the model numbers of the appliances you want and in short order the sales person is entering the order into the computer. “Okay, I’m supposed to ask you a few questions before we complete the order,” says the sales person. “What company built your house?” Well that’s kind of a strange question but, you answer the sales person question anyway. “Ohh, I’m sorry the new generation of appliances will not work in houses by that builder,” the sales person replies with a sympathetic expression. “What the hell” you think and you reply “Are you kidding?”

The sales person launches into this long winded explanation that basically explains that the new features that enable the appliances to cook faster, cool better and make you life easier will not work in your house. Sure you could still buy the appliances but don’t expect those advanced features to work and the price remains the same. Maybe in the future the new features might work but, I can’t really commit to that. The sales person sees that you’re not a happy camper and offers, “We have plenty of appliances with last year’s technology over here that will work in your house.”

What would you do? I believe one of Oracle’s greatest strengths is that the database, in its entirety, works across everything. I enjoyed my years working for Oracle on the delivery side of the business and often expounded to customers on how Oracle worked across most every operating system. It seems the tides are changing these days as the 11gR2 Flash Cache feature only works with Sun Solaris and Oracle Enterprise Linux. Both Sun Solaris and Oracle Enterprise Linux are owned and supported by Oracle. So if you went with another operating system the Flash Cache feature isn’t available to your business. So the question is, “Is Oracle chipping away at their strengths?” Hopefully, the Flash Cache feature becomes available across all the supported operating systems and this is a non-issue. The danger there however is that some customers are going to get the impression they are being pushed to the back of the line even after their sizable investment. So the question is for you, “What would you do?”

Oracle Flash Cache is a feature that allows the DBA to use enterprise flash disks (EFD) as a cache to improve database performance. Using Flash Cache a DBA defines the mount point, file name and amount of disk for the database to use in holding objects, like tables, for very fast access times. For more information on Oracle’s Flash Cache feature review these links:

http://guyharrison.squarespace.com/blog/2010/7/7/the-flash-cache-keep-option.html

http://download.oracle.com/docs/cd/E11882_01/server.112/e10595/memory005.htm#ADMIN13392

Posted in Uncategorized | Tagged | Leave a comment