TheSaffaGeek

My ramblings about all things technical


Leave a comment

VCDX Troubleshooting Skills

55020378So this posting isn’t about my opinion of if the dropping of the scenario is good or bad as in all honesty more time in the design scenario sounds great to me. This posting is actually about the resources I used to prepare for my VCDX troubleshooting scenario that I think an architect should know and thereby any good VCDX should also.

  • The first resources I used were actually the ones I used in my preparations for the VCAP5-DCA as this really makes you learn where all the logs are, what methods there are of troubleshooting issues and what you might be looking for. My study resources list for the VCAP5-DCA is a great start and if you are at the point of defending for VCDX you should have used some of these in your preparations but what I went over again were the troubleshooting videos by David Davis. Even though they are old the methods in them still apply especially ESXTOP etc.
  • The next resources were a mix between my two mentors for my recent VCDX attempt aka Larus Hjartarson and Rene van den Bedem. Both of them did brilliant breakdowns of how to prepare and think during the scenario and the methodology you need to keep to. These methods give you a great plan of attack even if it is a real world customer you are trying to help. Larus’ methodology is mention in his VCDX: Troubleshooting Scenario posting and Rene’s VCDX – Troubleshooting Scenario Strategy posting.
  • One resource that I felt was the best real world applicable resource I used that didn’t map perfectly to the VCDX scenario methodology but was brilliant was one that was recommended to me by Frank Buechsel who used to work for VMware GSS until recently was a book called Debugging—The Nine Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems. It’s more based at software development but each of the steps applies perfectly to troubleshooting any issues in a technology environment and now that the scenario has been stopped I can put out the loose outline from the book and kinds of questions I wrote up for each of the headings plus what I wanted to say to explain why i was asking in red that I wanted to ask in the scenario and how I thought it might fit:
    • UNDERSTAND THE SYSTEM
      • When did the problem start exactly?
      • What is meant to happen? – Why I am asking is because……………and what I’m hoping to achieve…………
      • When did you see the problem start happening? Is it recurring after a certain task or event or has it only happened once? – Why I am asking is because……………and what I’m hoping to achieve…………
      • Have any changes been made recently and are they tracked in a change management system? – Why I am asking is because……………and what I’m hoping to achieve…………
      • Have we collected logs or alerts from the systems and are we using something like vCOps where we can drill down and see alarms or alerts? ? – Why I am asking is because these mechanisms can give us ideas of the failures and possibly where it is happening if not just one location and what I’m hoping to achieve is to find the specific places the errors are showing, what the errors have been in the past if possible but also prepare for the next step of making it fail again so we can possibly see the error again or collect it for the first time.
    • MAKE IT FAIL
      • If it happens around a certain event can we try replicate the error and make it happen as often as possible? – Why I am asking is because I want to confirm the error is in fact happening at the point you mention and I’m hoping to achieve the exact step where it is happening and confirm if indeed our assumptions of when it is happening are true or not so we don’t waste time troubleshooting an assumption.
      • When are we doing the replication of the error can we document each step? – Why I am asking is because I want to confirm it is not just the step where it fails but the steps leading up to it in case a step in the sequence is then causing the eventual failure and I’m hoping to achieve the possible conflict or incorrect setting/step being followed.
    • QUIT THINKING AND LOOK
      • Are there any alarms or alerts on the source or destination system/s ? – Why I am asking is because I want to confirm not just the outcome of the failure that you mention but hopefully what is causing the failure and what I’m hoping to achieve is the point/component where we should do the troubleshooting so that we don’t make any unnecessary changes.
      • For the errors can we search the VMware/Vendor KB/Forums and see if any matches come up for some/all of the errors? – Why I am asking is because some of the errors might be known or even just give us an idea for where to look and what I’m hoping to achieve is to isolate the problem even more and not waste time looking at other components when a kb article might give us a good lead and save us precious time getting the issue fixed
    • DIVIDE AND CONQUER
      • For the machines that are failing are they the same configuration/going to the same location/coming from the same location/going over the same path? – Why I am asking is because I want to isolate the good parts/side and the bad parts/side and what I’m hoping to achieve is to focus my attention on the side that is showing the error so we don’t waste time and have less things to cover in the hope we can isolate the problem.
      • Can we try reverse the step in the opposite direction? – Why I am asking is because……………and what I’m hoping to achieve…………
    • CHANGE ONE THING AT A TIME
      • Try a migration/alteration/fix and if it doesn’t work then change it back and try something new. “Please can we migrate the failing machines to another host? “it still fails” Ok please move it back “– Why I am asking is because I don’t want to receive additional/red herring errors due to the change we made and what I’m hoping to achieve is to keep the environment unchanged as much as possible so we don’t cause additional errors/lose methods to troubleshoot.
    • KEEP AN AUDIT TRAIL (these were more writing out my thoughts and what I felt I needed to remember)
      • Write down what you did and the outcome and also WRITE DOWN THEIR RESPONSES as these may have clues!! “there are no errors in vSphere” might mean the error is not reaching vSphere for it to log the error so go “upstream” to find the source.
      • The error doesn’t sound like it is in vSphere so can we please look at the HBA on the host and ensure it is connected correctly and receiving data via ESXTOP.
    • CHECK THE PLUG
      • You state that the network connections are correct but please can we get it checked again? – Why I am asking is because I want to confirm that what we state is correct is in fact correct right now and what I’m hoping to achieve is to clear up any assumptions and have clear and confirmed facts about necessary “upstream” components.
      • Are the steps you are following worked in the past? Are we following the exact steps that worked before? – Why I am asking is because I want to confirm if it has ever worked/if we are following different processes and what I’m hoping to achieve is to confirm if it has ever worked and if a new step if causing the error to happen so we can troubleshoot what the different steps is bringing up.
    • GET A FRESH VIEW
      • Not really applicable to VCDX troubleshooting but asking for someone who is an SME in the customer might shed some new light/clear up what the exact problem is.
    • IF YOU DIDN’T FIX IT, IT AIN’T FIXED
      • Not really applicable to VCDX troubleshooting.

If you want to read about my utter joy about passing the VCDX then have a look at my VCDX #205 posting and also my VCDX Spotlight.

Next I’m hoping to dive deeper into each of the points from my VCDX #205 posting starting with VCDX Resources – Did you use them all??

Gregg

Advertisements


1 Comment

VCP6-CMA track on the EMEA #vBrownbag

Now that my VCDX defence is over and hopefully I’ve done enough to be awarded an illustrious number I can pick up my old responsibilities for the EMEA vBrownbag of which I am one of the hosts and decided that the VCP6-CMA track would be a great one to do.

african children dancing - Lets learn Some vRA goodness

We are going to be going through the blueprint in order so that it is easy to follow and watch as well if you are subscribed to the iTunes feed. So starting next week Tuesday at 7pm GMT (Clocks have changed in Europe so this is GMT not BST) we will be doing objective 1 and it is being covered by one of the co-hosts of the vBrownbag, Frank Buechsel. You can register for the series and future vBrownbag EMEA sessions via the following link https://attendee.gotowebinar.com/register/8540909933274906113 and can also subscribe to the vBrownbag YouTube channel.

 

If you are interested in covering an objective on the vBrownbag then please get touch as we are always looking for presenters.

 

Gregg


Leave a comment

VCDX Spotlight: Eric Shanks

Name: Eric Shanks

Twitter Handle: @eric_shanks

Blog URL: theITHollow.com

Current Employer: Ahead

VCDX #: 195

How did you get into using VMware?

I was at a Chicago Microsoft Users Group and a company called Altiris was speaking about virtualizing applications with their solution. The whole idea was pretty eye opening to me so when my boss suggested some enhancements to our infrastructure the virtualization concept was brought up again. After some testing we decided that VMware was the clear leader in the space so we virtualized our infrastructure on 4.0.

What made you decide to do the VCDX?

The VCDX certification was a challenge I wasn’t convinced that I could achieve, but I had to know for sure. A few other co-workers already had the credentials and I decided it was the time to find out what I was capable of doing.

How long did it take you to complete the whole VCDX journey?

The whole process took me about six months to complete. I already had my VCAP-DCA and VCAP-DCD before I decided to try the VCDX so that helped, but I left myself plenty of time to work out my design before submitting it.

What advice would you give to people thinking of pursuing the VCDX accreditation?

Talk with your family first about the endeavour. The VCDX is challenging, but more than that time consuming. Expect to spend nights and weekends working on it. The VCDX journey is personal achievement but can’t be done without some support from family, friends and co-workers.

Aside from talking with your family first, the second piece of advice I’d give is don’t be afraid to fail. This isn’t an easy challenge and many really qualified people have stumbled on it. It doesn’t mean you’re not awesome, it just means you need to tweak your design or presentation skills a bit.

If you could do the whole VCDX journey again what would you do differently?

I would have made sure to understand the defense blueprint better from the start. I felt that there were specific sections of the blueprint where I didn’t have enough things in my design to present. Specifically my design didn’t have a lot of “Security” related items so I wasn’t able to talk in depth about it in my defense. If I could have done it over I would have added an additional security requirement and supported it with my design so that I could talk about it in the defense.

Life after the VCDX?  How did your company respond?  Was it worth it?

After the process was over I was recognized at our company’s Tech Summit and given a bonus for the achievement. The feeling of a sense of accomplishment from meeting a personal goal made the process worth it.


Leave a comment

VCDX Spotlight: Nick Bowie

Name: Nick Bowie

Twitter Handle: @nickbowienz

Current Employer: ViFX Ltd.

VCDX #: 202

How did you get into using VMware?

I’d had exposure back in the earlier days with GSX and ESX Server through David Manconi (@dmanconi), when we worked together previously. I remember he was pretty excited about it, so I figured there must be something to it 😉 But I didn’t really have the opportunity to pursue it professionally until late 2009 when I became involved in a large data centre migration project which included establishing an SRM enabled ESXi 3.5 infrastructure.

What made you decide to do the VCDX?

I had heard about it through my go-to places like Yellow-Bricks, as I tried to soak up as much as I could about virtualisation. It wasn’t until I was fortunate enough to spend some time with Michael Webster (@vcdxnz001), who had just achieved his and, later in 2011, Travis Wood (@vTravWood) who was so excited to be pursuing it that I started to understand the level it represented. When I joined ViFX I realised the calibre of people I was fortunate enough to be working with and felt I really needed to lift my game.

How long did it take you to complete the whole VCDX journey?

It’s been such a gradual, incremental process it’s hard to pinpoint exactly when it started in earnest. If I measure from obtaining the VCP5-DCV in September 2012, it took almost three years. It became a realistic goal after joining ViFX in August 2013 though.

What advice would you give to people thinking of pursuing the VCDX accreditation?

It’s not a race 🙂 Each qualifying exam (VCP, VCAP DCA/DCD) are challenging in their own rights. Focus on the immediate, realistic goals and you’ll find your capabilities grow significantly through those achievements. When you’re at the point where you can submit the design: read the blueprint. Everyone says that, and I must admit I’d get a little frustrated at hearing it so often, but it’s true. Read it, understand it as best you can and make sure you touch on all the points.

If you could do the whole VCDX journey again what would you do differently?

I’m not sure I would do it too differently. The design I based it on was light on some components of the blueprint requirements, due to certain constraints and challenges, which meant some extra work was required on my own time. While working on that project I completed a few others in between that were more “blueprint-friendly”, in BC/DR and vBCA Oracle based engagements, but this was the one I had invested myself into with the goal in mind so I was determined to use it all the way through.

Life after the VCDX?  How did your company respond?  Was it worth it?

It’s only been a week, so it’s early days yet 😉 ViFX are extremely proud to have the 2nd VCDX in New Zealand, and the only one in the VMware partner space. We definitely have more VCDX’s in progress – watch this space!


Leave a comment

Four Ways Implementing a BC/DR Solution Can Help Your Small/Midsized Business

About six months ago I was approached by VMware SMB asking if i would be interested in writing a blog posting about how implementing a business continuity and disaster recover solution could help small and midsized businesses from my experiences as a senior consultant for Xtravirt. I wrote up the posting and the SMB team liked it so much they asked me if I would do a four part blog posting from the original blog posting and so I created a four part posting and the final part of the series was published yesterday. So below are links to the four blog postings and I hope  they are informative and beneficial to people. If you are unsure about any of the portions mentioned or want to know more, you can ask VMware Professional Services or Xtravirt who are a VMware partner, to come in and help you with the design/configuration of your environment.

 

Part4:  Virtual Machine Migration

 

I have also written a whitepaper for VMware SMB around the need for SMBs to rethink their virtualization infrastructure and to elaborate on a number of the features and solutions that are now available to SMB customers. Download the whitepaper to learn how these solutions can help your business and your customers.

 

Gregg


Leave a comment

VCDX Spotlight : Joe Clarke

Name: Joe Clarke

Twitter Handle: @elgwhoppo

Blog URL: www.elgwhoppo.com

Current Employer: Rolta AdvizeX

VCDX #: 138

How did you get into using VMware?

Started leveraging it when the company I worked for started a P2V project in 3.X days.

What made you decide to do the VCDX?

The desire to become the first VCDX at my company, and a little bit of notoriety as well.

How long did it take you to complete the whole VCDX journey?

Start to finish, it probably took me 6 months. Writing that documentation definitely took several full Saturdays at the office.

What advice would you give to people thinking of pursuing the VCDX accreditation?

DO IT. You will learn more than you ever imagined, pass or fail. Get ready for a rough period of time however, you will be consuming a lot of personal time.

http://elgwhoppo.com/2014/07/13/how-the-path-to-vcdx-will-change-you/

If you could do the whole VCDX journey again what would you do differently?

Thinking back, I probably would have perhaps relaxed on my documentation a little, but that’s only because I think I perhaps did too much. For example, I literally spelled out every single design consideration and rationale that I encountered with a full pros and cons matrix. That may have been a little over the top, but hey, it sure helped me make sure I knew why I had decided something.

Life after the VCDX?  How did your company respond?  Was it worth it?

The VCDX was a definite “market value adjustment” talking point in my yearly review. It also started a domino effect, and we now have 2 other VCDXs at the company in total, with several more studying currently for 2015 defenses. Completely worth it, highly recommend.


Leave a comment

VCDX Spotlight: Samir Roshan

Name: Samir Roshan

Twitter Handle: @kooltechies

Blog URL: http://thinkingloudoncloud.com

Current Employer: Nutanix

VCDX #: 124

How did you get into using VMware?

It’s been more than a decade of my adventure and journey with VMware and Virtualization. I started using VMware workstation as a customer in 2004 when I was working for Safenet. One day I got a call from a recruiter hiring for EMC2 in their project team which was working on VMware products. At that time VMware was not an independent company in India and it was still under the banner of EMC2. I was so impressed by Workstation back then that I decided to go ahead and joined them in 2005 then I moved from New Delhi to Bangalore. I spent around 8 years working in 3 different teams of VMware in two stints. The most recent was as a Senior TAM where I was leading the TAM practice for India and SAARC before joining Nutanix.

What made you decide to do the VCDX?

For me it was about the next level and being a better technical person in the journey. When I look back at my journey I think it was all well worth the effort.

How long did it take you to complete the whole VCDX journey?

The roots of my journey dates back to 2009 when VMware launched the advanced exams for VCDX. I was working in VMware R&D back then as Lead in the CPD team. My curiosity in the exam was due to the fact that it was the next level in VMware cert having achieved VCP and VCI I thought to conquer this exam. So I decided to take the exam on 3rd October 2009 and I got my results 17 days later with a score that I passed.

clip_image002

I was very serious about getting VCDX that I decided to leave my excellent job in VMware R&D and moved to IBM Software Labs in a consulting role. I knew that I can’t do VCDX without having a consulting/design experience. However I couldn’t pursue other necessary certification for next 2 years. Then in 2011 an excellent opportunity again knocked on my doors, I got a call for a position of Senior Consultant in VMware PSO when VMware started their consulting business in India. I grabbed the opportunity and started as the first consultant in the VMware PSO India team.

This was an excellent platform for me to start the journey again but at this time the VCDX was at version 4. I needed to take the same journey again so I cleared VCP 4, VCAP DCA, and VCAP DCD. I couldn’t focus on creating a VCDX proof design in that time period.

Then VCDX5 was announced and I thought lets do it on version 5, same process again, I cleared all the prerequisites and then started working on my design. This time I got it through; I paired with one of my colleagues Shankar Garikapati for the design and then we both submitted our design. It took 5 months from completing all the prerequisites, working on the design and then clearing the Defense.

What advice would you give to people thinking of pursuing the VCDX accreditation?

Don’t loose the sight on the goal, even though it’s the toughest and will take a longer time. Remember there is no glory in achieving something simple and timid. When you feel like quitting remember why you started.

I like the below quote by Mario Andretti

“Desire is the key to motivation, but it’s the determination and commitment to unrelenting pursuit of your goal – a commitment to excellence – that will enable you to attain the success you seek.”

VCDX is such a goal and you need to be prepare well, you have to pace yourself through the journey. You have to make sure that you are not burning yourself through the process it’s all about learning at each and very step.

If you could do the whole VCDX journey again what would you do differently?

I would not change anything much. I did what I could do best by keeping my work life balance. I didn’t burn myself and was able to spend time with my family and my newborn Son.

I was fortunate enough to pass in my first attempt and I am thankful to my family for that. Being the first VCDX in India was a special feeling for sure. I can’t forget the below message on 4th Nov 2013 by Mark Brunstad.

clip_image004

The first thing I did was to touch the feet of my Mother/Father for their blessings and thanked God (It’s an Indian thing you can say ;-)).

Life after the VCDX?  How did your company respond?  Was it worth it?

VCDX is worth every second, every minute every hour, every day that you spent on doing it. The VCDX process changed my perspective on many things, made me aware of my strengths and weaknesses. Made me a confident person, again it’s not the tag it’s the whole journey that’s matter.