Friday, March 10, 2006

Caution: Your VTL results may vary

By Beth Pariseau, News Writer
10 Mar 2006 |

Interest in virtual tape libraries (VTL) and disk-based backup in general exploded in 2005. But as with any new technology, when evaluation and acquisition give way to practical implementation - when, in a way, the "honeymoon period" ends -- users encounter new pitfalls they may not have expected.
One such case happened at the American Institute of Physics (AIP). According to James Wonder, director of online technology, AIP implemented a Sepaton Inc. S2100 VTL between its 13 terabyte StorageTek D178 SAN and Qualstar Corp. tape library last year. But when the new data-hungry disk system was attached to the back end of his approximately 40 Sun Microsystems Inc. servers running Solaris 8 and 10, however, he uncovered a new bottleneck in his backup system -- the servers themselves.

"The VTL pulled data out so fast, it increased CPU overhead on my production servers," Wonder said. "Like my Web server, for example, our production machines get hammered. We hadn't been able to roll the VTL out in big chunks like we wanted to, and it was affecting my customers."

Wonder's solution to this problem was to implement local replication by Kashya Inc. in which an image of each production server is mounted on a separate Sun backup server, which then sends the data to the VTL.

"That way, my CPU on a customer-facing machine doesn't get affected and my network load doesn't get affected writing to the VTL," he said. "We also use Kashya to clone to tape after writing data to the Sepaton VTL."

Greg Schulz, analyst and founder of the StorageIO Group notes that this user is probably moving his bottleneck around. "In general, whenever you move or remove a bottleneck, in this case the slow tape in the backup process, you invariably will shift or cause a bottleneck elsewhere, unless you have taken adequate precautions."

"We have seen VTL implementation expose 'new' bottlenecks," said Brad O'Neill, senior analyst with the Taneja Group. "The tape drive used to be the slowest part of the backup environment, but post-VTL, the customer discovers that their network or CPU or disk arrays are too slow to feed the VTL at maximum performance."

"When I was evaluating VTL solutions, I certainly did try to gauge what adverse impact, if any, was the host processor experiencing while trying to keep up with the enormous appetite for data of the virtual library," said Mark Stewart, backup and recovery storage administrator at Randolph Air Force Base in Texas and a user of FalconStor Software Inc.'s VirtualTape Library.

"On the vast majority of my hosts, there was no such impact," he added. But, he said, a collection of "antique" [Windows] NT4 systems have been difficult to back up with the VTL.

"I am forced to back up these boxes across the front-side LAN," he said, "And their connections are paltry, anemic 100 Mb fast-Ethernet NICs. The bottleneck there is the network connection, not an unduly high processor workload."
Newly networked storage using a VTL, as opposed to direct-attached tape, may also be an issue, Schulz said. "If you are moving data over a network, you may be encountering CPU cycles being consumed by TCP/IP processing as well."

"It cannot be overstated how vitally important it is to have from the very beginning a dedicated team of storage engineers familiar with every aspect of the entire backup environment evaluating this technology," Stewart said, "in order to be absolutely certain that there are no 'gotcha!' moments after you've already spent a large amount of money on a new piece of equipment."

The experts weigh in

"Even though VTLs can theoretically let you unplug your tape library and plug them in instead, that may not be the best way to go about implementing one," said Curtis Preston, senior analyst for GlassHouse Technologies Inc..

Other possible VTL pitfalls, according to Preston, include backup software licensing issues.

"Depending on your backup product, there may be pricing challenges with how the software is licensed for a VTL, as opposed to a tape library," he warned. Preston advises looking at pricing for VTLs by capacity rather than by tape drives or cartridges, which is how Veritas Backup Exec works.

"But other backup software vendors may still charge for licensing per tape drive or even tape cartridge. One thing lots of people like to do with VTLs is make lots of really small virtual tapes -- that's something they could wind up paying a whole lot more for until backup software pricing catches up."

Another thing to watch out for, Preston said, is that treating a VTL exactly like a tape library doesn't mean it won't work, "but if you acknowledge, at least when setting it up, that you're not actually writing to tape, there might be things you can do differently to get the best performance from the VTL."

For example, many storage administrators use a technique when writing to tape libraries called multiplexing, which is the splitting up of streams of data to push the most data through to tape.

"It's not something you need to do with a VTL, and if you continue doing it when sending data to disk rather than tape, you might defeat the purpose of a VTL, which is faster restores," Preston said.

In general, "look at your backup process and configurations, and see if the reason you set them up that way still exists."

Another note to Tivoli Storage Manager (TSM) users -- TSM doesn't multiplex, and most VTLs will support 32- or 64 virtual tape drives with some outliers at 96 or more. Meanwhile, it's common with TSM to be backing up hundreds of clients. Many TSM clients might require another disk pool in front of the VTL to prevent bottlenecks, according to Preston.

"Finally, if they weren't copying tapes before, and in my opinion most haven't been, and just have been sending originals off site," Preston added, "You can't eject tape from the VTL, so you need to figure out how to create a copy. It may be you have to buy a backup software module you didn't use before. It may come down to, 'I hope you're good at scripting.' "

"Another factor is what type of files, big or small, are being backed up -- as there is more performance overhead in opening and closing files -- so backing up lots of small files will have more of a pronounced impact," Schulz said

Added O'Neill, "Other factors include tweaking and tuning of the backup software. For example, the customer might need to adjust the block sizes via the backup software running on the backup server, which can make a huge difference, or turn off software compression."

Fujifilm Alerts Users to Dangers of ``Re-Certified'' Media Market; Investigation Finds Used Media Potential Source of Failures & Compliance Breaches

VALHALLA, N.Y.--(BUSINESS WIRE)--March 10, 2006--Fuji Photo Film U.S.A., Inc., a leading global provider of data storage tape cartridges, today announced the results of a study that found there may be serious risks to those companies buying and/or selling "re-certified" or used data storage tape media.

The Fujifilm study found that in addition to the risk of unintentionally passing along confidential company data, a large percentage of used media can be of questionable stability and often fails, making it unreliable for storing or archiving important data.

"Based on our findings, we would advise CIO's and IT managers to be aware of the risks associated with selling retired data tapes into the used market or using so-called re-certified tape in their own data centers," said Rich Gadomski, Vice President of Marketing, Fujifilm Recording Media Division. "In addition to buying media of unacceptable quality and performance, the uncontrolled practice of selling retired media can, in fact, allow your corporate data to end up in the wrong hands - potentially breaching corporate confidentiality policies and possibly violating government compliance regulations."

About the study

Fujifilm confirmed the dangers of selling and buying used media through a study conducted with Ovation Data Services, Inc., a leading provider of digital data management and data tape services with headquarters in Houston, Texas.

Ovation analyzed 30 "re-certified" LTO data tapes that were acquired on the open market. What Ovation found was that on many of the cartridges, while some initializing was performed on the initial section of tape to make the data inaccessible at the initial pass of a drive head, data in fact still existed on remaining portions of the tape.

Of the 30 tapes analyzed, eight tapes still contained data potentially recoverable through standard data recovery methods. In order to ensure existing data on a cartridge has been completely removed, traditional re-certifiers generally "degauss" the media. Degaussing is the process of "bulk erasing" data via exposure to a strong magnetic field.

While degaussing is an effective method on older tape technologies such as the early DLTtape format - which did not employ a magnetic servo track - it is not possible for newer tape technologies such as LTO and 3592 formats that do use magnetic servo tracks. These newer tape formats contain a factory-written, magnetic "servo track" that allows the finely tuned magneto heads of a storage drive to continuously monitor tracking for recording and reading of data across the hundreds of data tracks on the width of the tape. If a servo track is degaussed, the tape is rendered useless.

In order to fully erase existing data on an LTO format tape (or other tape utilizing a magnetic servo), it is necessary to "security erase" or completely overwrite the existing data across the entire length, a process that can take several hours. This process is not economically feasible for re-certifiers, and the results from the study indicate that, in fact, it was not performed at all for this particular set of cartridges.

The study also highlights the fact that there are no industry standards for "re-certifying" used media so, therefore, the quality and reliability of used media is questionable. While a company can request a detailed report on the condition, usage and age of the tape, this is rarely done. In addition to the eight tapes still containing data, two other tapes were so worn that the drives rejected the media outright (fatal load failures). Still other tapes had "tape alert flags," malfunctioning dust shutters, damaged gears, broken leaders or excessive debris.

Counterfeit tapes

Data center managers need to also beware of some resellers that are repackaging used tape and selling them as "new" in counterfeit manufacturer packaging. These counterfeit products are then sold to unsuspecting customers at a price below market value, usually via discount media or auction websites. Fujifilm has been made aware of several customers who found that they had purchased used tape sold as "new" tape, and continues to review its legal options when made aware of these illegal practices.

"Price competition and the availability of quality scanning technology have encouraged and enabled the counterfeit tape market," said Gadomski. "We are aggressively exposing this business as it may put our customers at risk and negatively impact the perception of our quality and value of our brand. It's important that companies realize that they may be contributing to the problem by selling their used tape - or by buying from un-authorized resellers and distributors."

Protecting your business

Whether you manage a large enterprise data center, or a small- or medium-sized business, the perceived cost savings from selling retired data storage media can be compelling.

The monetary gain from this practice, however, must be weighed carefully against the potential risks. Any data retention policy that causes a company to fall out of regulatory compliance can result in severe penalties that include fines, cessation of operations and/or criminal charges. CIOs and IT managers are encouraged to protect their businesses by immediately reviewing their media management policies from purchase to archive through to "end-of-life", ensuring that their companies:

1. Enact policies that ensure control of archived data before, during and after it leaves the data center and building;

2. Prohibit the buying or selling of used or recertified media to protect against potential breach of confidentiality policies or violation of government regulations;

3. Develop guidelines for confirming appropriate data deletion and destruction of retired media, or utilize third party companies that provide proper certification of destruction;

4. Specify "new, factory fresh" media when purchasing new media through a reseller. This will help to ensure that the media they receive is not merely re-certified or repackaged to appear as new.

Fujifilm actively works with partners, resellers and other manufacturers in the industry to ensure regulatory compliance is not compromised, and provides educational materials for CIOs and IT Managers that encourage "best practices" for handling, storing, transporting and disposing of data to protect a company's information assets.

For more information on Fujifilm data storage and other recording media products, please call 800-488-3854, or go to You can learn more about Fujifilm magnetic technologies at

Saturday, February 18, 2006

A Glimpse at Wiretap Device Central to the Case

By Charles Piller, Times Staff Writer

In a racketeering indictment issued last week, federal prosecutors contend that Hollywood private eye Anthony Pellicano helped develop a device called Telesleuth, which he then used in illegal wiretaps.

Telesleuth was used to convert voice recordings into digital files that could be stored on a computer for easier transcription, according to a spokesman for the U.S. attorney's office. Prosecutors would provide no other details.

Jill A. Cossman, an attorney with Greenberg, Glusker, Fields, Claman, Machtinger & Kinsella, represented Pellicano in an attempt to trademark Telesleuth in 1995. The application describes the product as "computer hardware and computer software which will be used for the monitoring and/or recording and subsequent playback of telecommunications."

The application was ultimately abandoned.

Bertram Fields, a prominent entertainment lawyer and partner in the firm, used Pellicano's investigation services and is a subject of the ongoing federal investigation.

Brian Sun, an attorney representing Greenberg, Glusker, called the idea that trademark attorneys would have known of or condoned criminal uses for Telesleuth "preposterous."

"It was specifically intended to be marketed to law enforcement officials," he said.

Pellicano was a high-profile investigator and expert in audio analysis for individuals, law firms and law enforcement agencies in criminal cases. He formerly operated a Los Angeles-based company called Forensic Audio Lab.

Today, recorders that turn spoken words into computer-readable files are inexpensive consumer products. But they were not widely available in 1995, when Pellicano allegedly hired software developer Kevin Kachikian to produce Telesleuth.

"I'd guess that Telesleuth was pretty low-tech, perhaps a self-contained digital voice recorder that could be physically attached to a phone line in the field," said Phil Karn, a telecom security expert in San Diego. "A well-designed device could draw what little power it needs from the phone line without needing a battery that would have to be replaced."

Experts said a sophisticated programmer could easily have set Telesleuth to decode the touch tones of the target's dialed numbers or to read incoming numbers detected by caller-ID services.

Kachikian pleaded not guilty to several charges in the Pellicano case, including destruction of computer files, hardware and software related to Telesleuth. He is free on $100,000 bail. Neither he nor his attorney returned calls seeking comment.

Kachikian's personal website features two products designed for Pellicano: Forensic Image Sleuth and Forensic Audio Sleuth, described as tools to verify the authenticity of image and sound files. Pellicano holds a trademark for Forensic Audio Sleuth.

In the 1990s, Kachikian also briefly worked as a consultant for the Los Angeles Times, producing software that enabled Macintosh computers to download weather data.

According to the indictment, Telesleuth was housed in a case Pellicano ordered from Philadelphia-based Amuneal Manufacturing Corp., which specializes in containers that block electromagnetic emissions. The case was designed to prevent interference from power lines, which are often near phone-switching boxes.

"A shielded case would avoid causing noise on telephone circuits, noise that would be annoying … enough that someone would call the phone company to complain," said Steven M. Bellovin, a computer scientist at Columbia University and former security researcher at Bell Labs. "That sort of investigation — purely technical, not for fear of wiretaps — would likely uncover the box."

Pellicano never told Amuneal why he wanted the cases, but ordered 24, said Jim Flannagan, a sales manager for the company. Flannagan said he vividly recalled the 6-year-old order, because Pellicano became incensed when the job was delayed a day. "He went ballistic" in a phone conversation, Flannagan said. "He called me a liar. I just hung up."

Experts said the alleged taps could have been placed inside the phone company computer system, outside the subscriber's home or office, or on phone lines in several locations between those points.

In the Pellicano case, the government described insider access as a key step. Field technician Rayford Earl Turner, who retired from SBC — now called AT&T — in 2001, allegedly obtained subscriber data on Pellicano's targets from Teresa Wright, a sales support manager with SBC, California's largest local phone company.

Wright worked in the company's Wilshire Boulevard office until she was fired in November 2003. She pleaded guilty to fraud associated with Pellicano's alleged wiretaps.


Times staff writer Andrew Blankstein contributed to this report.

Saturday, February 11, 2006

Archive To Survive

February 10, 2006
By Drew Robb

Archiving is certainly one of the hottest areas of storage right now. It lies at the heart of a subject that is near, though perhaps not so dear, to the heart of every storage manager — retaining a legion of records to comply with an ever-expanding roster of regulatory requirements.

"Compliant records data is presently estimated to grow over 60 percent per year, generating more than 1.6 PB of new storage capacity requirements in 2006," says Fred Moore of Horison Information Strategies. "This represents the single fastest-growing application segment of the storage industry."

Part of the appeal may lie in the sheer scope of the topic. Ask some vendors what archiving is and they wax lyrical about disks. Ask others and it's all about tape. Bring up the subject at a conference and you might hear about the problems of e-mail storage or the compliance demands of the modern enterprise. And yet other views may cover the importance of search capabilities, the role of instant messaging (IM) in archive policy and how archiving is a central element of any information lifecycle management (ILM) strategy.

That's why the vendor acquisition frenzy surrounding archiving may seem a little confusing at first glance: OTG acquired by Legato in 2003 and subsequently gobbled up by EMC; Educom picked up by Zantaz; KVS by Veritas and then Symantec; Computer Associates' acquisition of ilumin Software Services; Symantec purchasing IMlogic; and Quest purchased AfterMail, to name just a few. These buy-outs forward vendor roadmaps encompassing a wide range of archiving products and possibilities.

"All the various archiving options such as active archiving, ILM and tape have their place," says John Webster, senior analyst and partner at Data Mobility Group.

Finding its Centera

As the array king, it's no surprise that EMC is one of the major champions of disk-based archiving. It advocates a three-pronged attack on the archiving space: smart/fast storage and retrieval at the high end, general array-based archiving in the middle, and tape library usage at the bottom of the pile.

"Centera is the choice if online access is needed with assured content authenticity at a TCO lower than a tape library," says Mark Avery, senior director of EMC Centera. "If all that is needed is speed to information, then CLARiiON may be the appropriate solution. If neither speed to information nor assured content authenticity is needed, then ADIC tape technology is an adequate solution."

Via Centera, EMC is going all out with its concept of authentic active archiving — archived information available on-line while assuring content authenticity. This comes under the broader heading of content addressed storage (CAS).

Centera is a purpose-built hardware and software storage platform for information archiving. CentraStar, Centera's operating environment, checks the authenticity of content, provides retention policy management capabilities and does single instancing, the storing of unique content only once. This platform has been strengthened with search and chargeback reporting capabilities based on technology OEM'ed from Norway-based Fast Search & Transfer ASA (FAST). CenteraSeek also allows fast cross-application advanced search of storage-based content metadata.

"Customers want to unlock the value in their long-term content, and that does not happen by storing it on tape or optical technology," says Avery. "With Centera, customers can afford to have disk that spins all the time, because it costs them no more and often less on a TCO basis than tape."

While EMC champions disk-based archiving, not everyone is convinced of its TCO claims.

"Long-term archival storage remains the realm of magnetic tape libraries," says Moore. "Today there is no truly cost-effective ILM strategy without a tape component."

Save Our Spam

Compliance is obviously a major driver of the market, perhaps more so in e-mail archiving than anywhere else.

In the past, storage managers were forced to restore and pull data from backup tapes in a costly and services-driven exercise that exposed firms to regulatory sanctions and unsuccessful litigation outcomes.

With new message management technology, there are now more cost-effective and efficient solutions to the problem. The market for messaging archiving continues to grow, driven in large part by the focus on regulatory and legal requirements to retain and discover e-mail.

"In 2005, we saw a large increase in demands for e-mail archiving as case law and regulatory pressures continued to force companies to retain e-mail as business records," says Mike Gundling, CA's vice president of product management. "Organizations are reeling from high-profile litigation and important legal priorities that are driving the need for e-mail discovery."

CA bought ilumin to fill out the archiving features of its BrightStor storage management line. Known as CA Message Manager, it offers advanced compliance and litigation support for enterprise e-mail archiving. Thus CA's strategy is to become a hardware agnostic one-stop shop for e-mail security (eTrust), backup (ARCserve) and archiving (CA Message Manager).

CA treads the middle ground between tape and disk. In its view, both play distinct roles. However, it reports that over 100 of its customers using optical jukeboxes have switched to disk-based WORM. Further, CA believes that active archiving is not a replacement for traditional backup.

"Organizations need to understand that active archiving is everything a backup solution cannot be," says Gundling. "But they need both archiving and backup."

Another vendor making a play in the e-mail arena is Mimosa Systems, based in Santa Clara, Calif. Mimosa NearPoint provides immediate mailbox and message recovery, disaster recovery, e-mail archiving and self-service search and access in one solution. NearPoint is disk-based, using SATA RAID and NAS appliances. It runs on a Windows 2003 server.

"Customers have realized that first generation e-mail archive products did not deliver value, so the e-mail archive market is now white hot," says Bob Spurzem, senior product manager at Mimosa.

The relevance of advanced e-mail functionality can be seen in the findings of a recent survey conducted by Osterman Research. E-mail management is front and center among enterprise concerns, with the top Exchange-based problems being the management of e-mail disaster recovery, the sheer size of the message stores, protecting e-mail databases and searching individual files for legal discovery.

"Enterprises are facing serious challenges across multiple areas of e-mail management," says Michael Osterman of Osterman Research. "The market is demanding solutions for holistic e-mail data management that go beyond a single feature or function."

Storing or Accessing?

The axis of attention in archiving, then, appears to have shifted. The traditional methodology — dump all those old tapes in a closet or onto a rickety old array — is under attack.

"Archiving is not about storage, it's about access," says Michael Howard, chairman and CEO of OuterBay Technologies, which was acquired earlier this week by HP.

Like EMC and CA, his company is talking up active archiving — but with a Very Large Database (VLDB) slant. The generally bloated state of databases, it seems, has gone to a whole new level of late. OuterBay reports customers with multi-terabyte or even multi-petabyte databases. For example, one VLDB customer has DB tables of one terabyte. Another archives 3 TB of database transactions per year.

In response, the company has released OuterBay Enterprise Edition and Compliance Edition. The latter is a self-contained XML archive with added audit capabilities, data integrity features, data lineage and WORM storage device integration.

"We're seeing more and more customized applications in enterprises with large scale databases," says Ray Paquet, an analyst with Gartner. "Enterprises should look to database archiving applications to manage accelerating growth, increase enterprise application performance and meet compliance requirements."

Disk Head Debate

Many of the solutions referenced in this article advocate disk-based archiving. Yet the popular wisdom is that disks should not be used for long-term storage as the disk heads could stick. Copan, for one, claims a solution to the problem with its Disk Aerobics technology, which powers up disk drives once a month to keep them active and reduce failures.

"It is true that disk drives are not well suited to be unplugged and stored on the shelf," says Spurzem. "Long-term data storage is better suited for tape, and to optical if the capacities are a fit for the application."

He believes, though, that tape and disk can be combined into a cost-effective combination for backup and archiving. Disks supply the near-term storage and tape deals with more lengthy retention requirements.

The role of tape in the archiving hierarchy is being further strengthened by the latest feature sets. Tape technologies such as LTO and SDLT offer increased capacity and better security. Encryption, in particular, is seen as a major step forward in the usefulness of tape as an archiving medium.

"Encryption is making its way into the tape designs, positioning the tape industry for an explosion in compliance and archival applications," says Moore.

For more storage features, visit Enterprise Storage Forum Special Reports

Work begins on ASU forensics facility

By Deborah Willoughby
Montgomery Advertiser

Ground was broken Friday for a new $10.5 million building at Alabama State University that will help solve crimes and train a new generation of forensic scientists.

ASU President Joe A. Lee, who defined forensic science as the application of sciences to the law, said there's a great need for scientists to solve crimes.

"There are church burnings in Alabama right now," Lee said, talking about how physical evidence left at the scenes of church arsons, including tire tracks, may help break the cases.

Students and officials at the groundbreaking for the 50,000-square-foot Forensic Sciences Facility focused on the partnerships of state forensics employees working in the same building as ASU chemistry and biology students.

The building, which will be completed in 18 months, will have state laboratories, a mock courtroom, an instrumentation laboratory and a criminal logistics lab. It's a joint project of the state Department of Forensic Sciences and ASU, mostly paid for by voter-approved bonds, with $3 million coming from state funds.

"It will provide the latest in technology for our state laboratories, but will also create a trained pool of potential employees that we need in this critical area," said F. Taylor Noogle, director of the state Department of Forensic Sciences.

Jennifer York, an Alabama State student from Selma, was impressed by plans for the new building. York, a junior studying business administration, said, "This building will bring more students and more opportunities to the university."

ASU also had its 106th Founder's Day convocation on Friday.

Enter your email address below to subscribe to FORENSIC FILES: Audio. Video, Training Technology & News!

powered by Bloglet