This is my cyberhome!

June 4, 2004

Verifying a VSAM file

Filed under: Mainframes — Manish Bansal @ 8:50 am

When a VSAM file is opened by a job in the output mode, a flag in the VSAM catalog called “open-for-output” gets set to ‘ON’. This flag does not get turned off until the job ends successfully i.e. until the job closes the file normally. Or in case we are editing the file manually in file-aid, the flag gets set when we open the file and gets turned off when we close it. But say the job goes down halfway through the update process or our TSO session expires before we could close the file. The “open-for-output” flag remains turned on. What happens to the file now?

When next time a job/user tries to open the file for output, the file manager trots off to the catalog to turn on the flag. But the flag is already on!! So it guesses (somewhat optimistically) that some other job might have opened the file. It issues a GRS (Global resource serialization) enqueueing on the file to find that out. If the file is not open anywhere else, it comes to know that it was not closed properly during its last use and its time for some catalog cleanup. Enter VERIFY.

Verify is a record management macro like get or put. In our case, the open processing issues an implicit verify against the file. This can be confirmed by the IEC161I type warning messages (RC of 56 and 62) in the sysout. The verify macro will compare the ICF catalog information with the physical VSAM cluster. It starts reading the VSAM dataset CI by CI, starting with the High Used RBA. It compares HURBA value, number of index levels, system time-stamp and many other fields. If the verify is not successful, it will issue a warning message with a return code of 116 (X’74’). Two things worth noting about implicit verify macro:

  1. It will not correct the catalog information. That is the job reserved for IDCAMS verify. It will just issue some warning messages. It is up to us to figure out what to do next. We can continue processing but the data integrity won’t be guaranteed.
  2. Implicit verify is not issued if the file is being opened for input or reset processing. Or if the VSAM file is of type LDS.

IDCAMS verify is an explicit verify command. When an IDCAMS verify is issued against a file, it opens the file, calls the record management verify macro and then closes the file. And it is at the time of closing the file that the ICF catalog gets updated and the “open-for-output” flag gets reset. Two things worth noting about explicit verify macro:

  1. A successful verify does not guarantee that everything is fine now. The catalog statistics may be invalid; the file might have duplicate or missing records. HURBA may be off by a few bytes.
  2. IDCAMS verify does not update the catalog or the VSAM control blocks directly. It relies on the implicit verify and VSAM close processing to do its job.

So whether the verify is successful or not, the structural integrity of a VSAM file is not guaranteed. Its always a good idea to take standard recovery actions on such files.

Advertisements

40 Comments »

  1. Hi,
    Do you recomment to Verify the VSAM in the Job before opening the file in OUTPUT or I-O Mode??

    Thanks..
    Chandra

    Comment by Anonymous — October 26, 2004 @ 7:39 am | Reply

  2. It’s always a good idea to issue an exclipit IDCAMS VERIFY on the file before opening it for OUTPUT. And don’t forget to query the RC out of that step.

    Comment by Manish Bansal — October 26, 2004 @ 6:29 pm | Reply

  3. Hi,
    I have question which relates to VSAM file.

    I delete defined a VSAM file(KSDS) and tried to access thru FileAid BROWSE option and its failed saying that “File is empty”.I tried to READ the file using a program too that too found to be not encountering the ENDOF FILE condition.

    Then I inserted a dummy record to file using FIle-Aid in EDIT mode.Then Deleted the DUMMY record just I have inserted to file.

    Now I repeated the first steps I mentioned (using FIle Aid tried to Browse and using the Program tried to READ the file) suprisingly..instead of showing “File is empty ” from FileAid it it opened the contyent window and also the END OF FILE condition occured in Cobol program too…

    Can u tell why its is so?

    Thanks in advance

    Karthik

    Comment by Karthik — July 27, 2006 @ 5:01 am | Reply

  4. Hi Karthik,
    When you create a new VSAM file, the HURBA is set to zero. HURBA stands for Highest Used Relative Byte Address. When HURBA is zero, you cannot open the file for input. When you insert a record in the KSDS file and then delete it, the HURBA does NOT get set to zero again. So you are then able to open the file in file-aid.
    However, if the VSAM file was defined with REUSE option, the HURBA does get set to zero after you insert/delete the record.

    Comment by manishbansal — July 27, 2006 @ 5:43 am | Reply

  5. HOW TO FIND THE LAST RECORD IN KSDS FILE?

    I MEANS TO ASK I WANT TO TRACK THE ONLT LAST RECORD OF KSDS OR ESDS FIEL.

    PLEASE LET ME KNOW THE ANSWER IF YOU KNOW.

    Comment by RAMESH — August 4, 2006 @ 10:28 am | Reply

  6. Hi Ramesh, can you please tell me some more details? Are you trying to do that through a cobol program? What exactly you want to achieve? And what problem you are facing now?

    Comment by manishbansal — August 4, 2006 @ 10:52 am | Reply

  7. Hi Manishbansal,

    Nice to see your mail,
    Regarding the question i want to ask you is

    (1) 2 input vsam files files, one is ksds, second one is esds, both files contains the data.

    (2) output file is flat file, should contain the last records from both the input file, this is to perform through cobol program.

    pleas let me know the answer.

    Thanks again,

    Comment by RAMESH — August 9, 2006 @ 5:11 am | Reply

  8. I still havn’t understood the question properly. Are you trying to merge the two files and then take the last record? Or do you want the last record from each file?
    Anyway, you can get the last record by moving High-Values to the key, doing a Read-Next and then doing a Read-Prev.

    Comment by manishbansal — August 9, 2006 @ 12:02 pm | Reply

  9. Manish, it seems be right answer.

    thank you so much.

    Comment by RAMESH — August 10, 2006 @ 5:54 am | Reply

  10. What is the use of NOIMBED and NOREPLICATE options in define cluster?

    Please let me know.

    Comment by RAMESH — August 10, 2006 @ 5:56 am | Reply

  11. Hi,

    //EW238001 EXEC IDCAMS,SCL=H
    //DD1 DD DSN=FILE1,DISP=SHR
    //DD2 DD DSN=FILE2,DISP=SHR
    //DD3 DD DSN=FILE3,DISP=SHR
    //SYSIN DD *
    VERIFY FILE(DD1)
    VERIFY FILE(DD2)
    VERIFY FILE(DD3)
    /*
    //*
    //***************************************************************
    //*
    //* ABEND WHEN CONDITION CODE OF PREVIOUS STEP GREATER THAN 0
    //*
    //***************************************************************
    //*
    //EW238002 EXEC PGM=SUT302M0,PARM=’0111′,COND=(0,GE,EW238001.IDCAMS)
    //*

    The job abends with condition code 0111. That means that step EW238001 did not execute successfully.

    Can you tell what could be the reason? I read your article on IDACAMS VERIFY function but, still I could not figure out the problem with my job.

    Comment by Biman — October 26, 2006 @ 11:33 am | Reply

  12. What is the use of NOIMBED, NOREPLICATE parameters in Define Cluster, I see the same question has been asked by Ramesh also on Aug 10th. Can you please help me in understanding it.

    Comment by Nagalakshmi — October 26, 2006 @ 11:35 am | Reply

  13. Replicate:
    A KSDS VSAM cluster has two parts – Data and Index. When we access a VSAM record using a key, the system has to first read the index and get the address of the data record from there. As with any DASD access, the disk platter has to rotate before the desired index entry comes under the magnetic head. This time lag is called rotational delay and is typically measured in milliseconds. In the days gone by, the hard disks used to be very slow and rotational delay could be as much as 10-15 milliseconds. To minimize this delay, each index entry is replicated or repeated as many times as would fit on a track. This insures that no matter where the magnetic head is on a platter, the disk has to spin very little to read the desired index.

    Imbed:
    The other type of delay that occurs during a hard disk read is called seek time. This is the time taken for the magnetic head to position itself over the desired track. To minimize this delay, the sequence set portion of the index is replicated on to the first track of the data CA (Imbed). And each sequence set index points to the next data CA and so on. This minimizes seek time only in case of sequential access. Once the magnetic head has been positioned over the first tack and sequence set retrieved, the head assembly does not have to move again. Only the disk has to spin to read the next record and so on. This option does not improve performance in case of random access.

    Note:
    Both these options are no longer supported because the hard disks nowadays are fast enough to not warrant such measures. Even if you do specify them while allocating the VSAM cluster, they would be ignored.

    Comment by manishbansal — October 27, 2006 @ 5:53 am | Reply

  14. Thank You So Much Manish. I need to remove these options in the JCL and the programs. Beforing removing it, I wanted to know the use of it.

    Comment by Nagalakshmi — October 31, 2006 @ 1:14 pm | Reply

  15. Hi,

    How to set the HURBA value of the VSAM file in TSO and through the batch job?

    I have one VSAM KSDS file whose records RBA is not in a sequence. Record length of the file is 1000 bytes. So in this case the RBA of the first record should be from 0 to 999, then for the second record it should be 1000 – 1999 and so on. But it is not that way. RBA values are not in sequence. Due to this I am not able to access the record sequentially.

    Let me know what could be the reason behind this?

    Regards,
    Swapnali

    Comment by Swapnali — December 28, 2006 @ 5:36 am | Reply

  16. Hi Swapnali, I am not able to understand your question. If you have a VSAM KSDS file, what is the role of RBA there? You should be accessing this file by its key.
    Regarding HURBA, you don’t have to set it manually. It gets set automatically when you write a record to it.

    Comment by Manish Bansal — December 28, 2006 @ 9:03 am | Reply

  17. Hi Manish,

    Thanks for your reply.

    Yes, I can access the VSAM KSDS file by its key. But the requirement was to access the VSAM KSDS file sequentially and the file which I was trying to access, has the RBA out of sequence.

    Right now I have created one more file of the same structure and copied all the records from the affected VSAM file to the new file, by using REPRO. New file has the RBA in sequence.

    I just want to know that why RBA got out of sequence in OLD file?

    Comment by Swapnali — December 29, 2006 @ 8:41 am | Reply

  18. Hi Swapnali, I do not know the exact reason for this but it typically happens when a job abends while the file update was in progress. Or if you don’t close the file at the end of the program, the system will try to close it for you but sometimes it fails and the file gets corrupted. Try doing an IDCAMS VERIFY on this and see what comes up.

    Comment by Manish Bansal — January 7, 2007 @ 3:22 pm | Reply

  19. i want to read a ksds file from the last record because to update the same file i need the count of the file which is written one day before

    Comment by jagadesh — April 4, 2007 @ 6:03 am | Reply

  20. In my Job a step does a REPRO for a VSAM file but when VSAM file is empty it abends with RC=12. I think of a add a IDCAMS VERIFY before that step. So if the file is empty i can skip the main step and job runs fine..But i dont know what are the RC of IDCAMS VERIFY when file is empty or there is data in VSAM file.

    Comment by Anurag Johri — July 16, 2007 @ 2:00 pm | Reply

  21. Hi Manish,

    Can you please advise, how to read a vsam file using partial key.

    Comment by Sridhar — August 16, 2007 @ 9:45 am | Reply

  22. Hi,

    Can anyone explain me what is the meaning of SCL parameter in the IDCAMS step. I have never seen this before. Here is an example:

    //XW931003 EXEC IDCAMS,SCL=’*’
    //SYSIN DD *
    DELETE PCC1.XW9310C0.NDM
    DELETE PCC1.XW9311C0.NDM

    Thanks,
    Jasmine

    Comment by Jasmine Kar — October 1, 2007 @ 4:57 pm | Reply

  23. File1 – VSAM
    File2 – VSAM
    File3 – Flat
    What i have to do is,
    Read flat file Search that record in VSAM files If found then update Flat file with one field in VSAM file.

    Comment by Sudip — January 3, 2008 @ 5:10 am | Reply

  24. can u help me ……… i need a cobol programme

    Comment by Sudip — January 3, 2008 @ 5:12 am | Reply

  25. hi
    i having error in idcams which showing return code 16 while running the jcl,

    Comment by maridurai — May 9, 2008 @ 9:25 am | Reply

  26. hi, can i have the code to find the last record in a vsam file?

    Comment by anu — October 22, 2008 @ 9:48 am | Reply

  27. how can i see the last record in vsam file while i am in reading the records?

    Comment by SELVAPRIYA — October 29, 2008 @ 9:11 am | Reply

  28. How to identify a record in KSDS file using the partial key??

    Comment by Appu — December 13, 2009 @ 2:41 pm | Reply

  29. Working with Newly created VSAM.
    U’ll get RC 35 if u try to read/write, or RC 12 for verifying newly created VSAM through batch job.
    The trick is to open, add 1 record and delete it manually.

    at my place, we use KSDSINIT program (which do that trick using assembler or something – i dont who the creator is, i doubt that it is provide by the system since i found nothing when i googling KSDSINIT keyword )

    //KSDSINIT EXEC PGM=KSDSINIT,COND=(0,NE)
    //STEPLIB DD DSN=SYS1.LINKLIB,DISP=SHR
    //KSDS DD DSN=MY.DSN,DISP=OLD
    //SYSPRINT DD SYSOUT=*

    or you can use second alternative, put low-values after you define VSAM. But of course, your must ensure to ignore that low-values in your program.

    //BSLSREP EXEC PGM=IDCAMS,COND=(0,NE)
    //SYSPRINT DD SYSOUT=*
    DELETE INTLX.BSS.BSLSREP PURGE CLUSTER
    IF MAXCC = 8 THEN SET MAXCC = 0
    DEFINE CLUSTER –
    (NAME(MY.DSN) –
    INDEXED –
    SHAREOPTIONS(2,3) –
    NOERASE –
    RECOVERY –
    NOWRITECHECK –
    NONSPANNED) –
    DATA –
    (NAME(MY.DSN.DATA) –
    KEYS(15 0) –
    CONTROLINTERVALSIZE(4096) –
    CYLINDER(2 1) –
    FREESPACE(0 0) –
    RECORDSIZE(30 30) –

    ) –
    INDEX –
    (NAME(MY.DSN.INDEX) –
    CONTROLINTERVALSIZE(4096) –
    CYLINDER(1 1) –
    NOIMBED –
    UNORDERED –
    NOREPLICATE)
    /*
    //**——
    //WRITORMS EXEC PGM=DITTO,PARM=’JOBSTREAM’,COND=(0,NE)
    //SYSPRINT DD SYSOUT=A
    //OUTFILE DD DSN=MY.DSN,DISP=SHR
    //SYSIN DD *
    $$DITTO BV OUTPUT=OUTFILE,RECSIZE=30,NLRECS=1,FILLCHAR=X’00’
    /*

    in my batch program:
    READ MYDSN
    AT END MOVE ‘Y’ TO WS-SWITCH-END-OF-FILE.

    IF MYDSN-KEY = LOW-VALUE
    READ MYDSN
    AT END MOVE ‘Y’ TO WS-SWITCH-END-OF-FILE.

    Comment by eula — October 21, 2010 @ 2:01 pm | Reply

  30. i forget to say thank you to manishbansal for the article (this one and directory blocks).
    keep on sharing 😀

    Comment by eula — October 21, 2010 @ 2:19 pm | Reply

  31. Hi,

    Can you tell me if there is a way to verify the internal control structures of a VSAM file. Before, we used a tool called VSAM Mechanic, which did an outstanding job fixing the broken RBAs, etc. Is there a facility in IDCAMS that can do this, as we may no longer have this tool in shop?

    Regards,
    Anthony

    Comment by Anthony — November 11, 2010 @ 10:00 pm | Reply

    • IDCAMS VERIFY command should be able to do it. I don’t think it can repair any corruption but it can certainly verify the file integrity.

      Comment by Manish Bansal — November 20, 2010 @ 11:40 am | Reply

  32. Hi Manish,
    I am using a COBOL program to open a VSAM file in I-O mode. It fails with status 90. I recreated it with shareoptions 3,3. Still does not seem to make any difference. I am pretty unsure of how to proceed from here. Any thoughts will be welcome.
    Thanks much.

    Comment by surya — March 4, 2011 @ 2:46 pm | Reply

    • Hi Surya,
      Sorry for the late reply. It would be tough for me to answer this without knowing more details about the job setup etc.

      Regards,
      Manish.

      Comment by Manish Bansal — March 13, 2011 @ 9:47 pm | Reply

  33. Hi Manish,
    Can i verify a vsam file which is open as read only in CICS?

    Comment by Ivan H — March 14, 2012 @ 8:15 am | Reply

    • @Ivan,
      I am not 100% sure but it should be possible as the VERIFY command works at the system level whereas the read-only mode is valid only for the program which is using that file. In other words, the VSAM file itself has no read-only or read-write mode at the system level.

      Comment by Manish Bansal — March 14, 2012 @ 7:50 pm | Reply

      • Thanks Manish,
        To elaborate the scenario – I have a file “A” which is open in CICS with the FCT set as “READ ONLY”. I try to execute a TSO / IDCAMS VERIFY on this file and receive an error message “Dataset already in Use, try later”. This message would make more sense if this file was updatable in CICS. Any clue on this one?

        Comment by Ivan H — March 21, 2012 @ 8:34 am | Reply

  34. how read ksds file dynamically and write after 10 records from ksds file to flat file.?

    Comment by Anonymous — April 18, 2012 @ 6:25 pm | Reply

  35. Thank you Manish Bansal for the good article

    Comment by ankoo — May 7, 2012 @ 11:04 am | Reply

  36. Greetings I am so grateful I found yohr website,
    I really found you by error, while I was searching on Bing for something else, Anyways I am here
    now and would just like to say many thanks for a fantastic post and a
    all round thrilling blog (I also love the theme/design), I don’t have time to look
    over it aall at the moment but I have saved it and also added
    in your RSS feeds, so when I have time I will be back to read much more, Please do keep up
    the great b.

    Comment by Facebook Web Development — July 1, 2017 @ 2:15 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: