Little Plastic Castle

 
    
26
Aug 2008
3:33 AM PDT
   

Tuesday 8/26/2008

10:15 - I wanted to start working on the RHEL5 gold image today.� VIC in New York didn't give me a 5 option until I moved over to Mage.� Mage has 3.02 on it, Seer only 3.01.� 5 Wasn't an option on 3.01.

- Kipper needs me to look @ gargoyle.

� - A few weeks ago Kipper called saying that he couldn't get processes restarted after a change window on cyclops & gargoyle.� It can't spawn enough processes he says.

2008-08-26-09:46:05.742-07:00I----- 0x38AD51C2 webseald ERROR wiv pthread WsWorker.cpp 495 0xb5c06080
DPWIV0450E Could not create new thread (11)
2008-08-26-09:46:05.814-07:00I----- 0x38AD50A4 webseald WARNING wiv general IVServer.cpp 978 0xb5c06080
DPWIV0164W Could not start background process

Explanation:
WebSEAL failed to create an additional thread. This may be due to running out of operating system resources or exceeding process limits.

Action:
Check memory and thread limits for the process, and available memory. The error number can be looked up in /usr/include/sys/errno.h for more details on the problem.

- I raised the nproc limit in /etc/security/limits.conf a few weeks ago, set the hard limit up from 1000 to 3000.� Today I set the soft limit up from 400 to 1000.� No difference.� PMR is open with IBM.

0 - Got the OK to boot clover and basil last night.� It turned into a complete clusterfuck that was my fault entirely.

� - A few weeks back, Slick deployed these VMs and did the identity change and tool install etc.� He's not a UNIX person, but he knows his way around well enough.� After he finished, he asked me to double-check what he'd done and I completely blew him off because I was working on other things.

- The servers rebooted, but came up complaining about VMWare Tools not matching the running kernel.� I think Slick compiled VMWare tools before shutting down, not after restarting.� So when he did the compile, he was still on the old kernel.

- I booted them both back to their old kernel and got networking up, then loaded in a compiler and tried to find the kernel headers or source code on RHN.� It wasn't available and I have no idea why.� Perhaps the kernel was retracted, I don't know.� At any rate, I finally got the source code from kernel.org and got it through our firewalls and into a system where I could move it around.

- Right in the middle of this, a storm blew through the desert and knocked down my business Internet connection but spared my home connect even though they're both in the same house.� grrrr.

- Booted the systems to the new kernel, compiled the tools, unloaded the source & compilers and turned them back over to Becky to get OV up and running.

Ended up taking 1.5 hours instead of 20 minutes.

tada!

Add Comment:

Add Tags:
To add multiple tags, please separate them with comma ( , )





coraline's Profile

  • Username: coraline
  • Gender / Age: Female, 54
  • Location: USA - Arizona
  •  
     
     
    CORALINE's Interests:

    About Me: Little Plastic Castle is an extension of This Child's Life which is not hosted at inboxjournal. This blog will make little sense to anyone other than myself. It's a stream of consciousness log of what I do on a daily basis in my job as a UNIX computer systems engineer. o/~ "They say goldfish have no memory, I guess their lives are much like mine...so the little plastic castle is a surprise every time, and its hard to say if they're happy, but they don't seem much to mind." o/~ I endeavor to respect confidentiality and thus I use nicknames for the people I work with. (Mr. T, Ratface, Jerry, Cartman, Grandpa Simpson, Slick, and so on.) I drop system names in here and abbreviations, but any note that requires something like a FQDN will always be marked private. You can comment if you like, but I'll only respond if I feel like it.

    Interests: Cast of Characters: Just a few of the names I use for people, this is the nickname and what their job responsibility is. Mr. T - My Direct Manager (Engineering) Slick - ESX Specialist Cartman - Network Engineering Lead Grandpa Simpson - Network Engineer Becky - WebSphere Engineer Ratface - Developer/Architect Jerry - Project Manager (BTI) Matt T. - Project Manager (Blackbird) The Martian - Application Developer AVP Ice Queen - Engineering AVP Chez - Assistant to Ice Queen Watts - Engineering/Development Sr. Mgr. Hoppy - I.T. Security LDAP Squirrelface - I.T. Security Accts Woodchuck - I.T. Security Accts Russian Bride - Application Tester Trisha - Testing AVP Tequila - UNIX Admin OPS Chops - UNIX Admin OPS Mac - UNIX Engineer (Contractor, works with Tequila & Chops) Margie - SAN specialist (ch)InkPen - Backup Specialist Chimpy - OPS Mgr Duke - UNIX Lead even though he knows nothing about UNIX