This is my cyberhome!

Programming, Photography and more…

Verifying a VSAM file

When a VSAM file is opened by a job in the output mode, a flag in the VSAM catalog called “open-for-output” gets set to ‘ON’. This flag does not get turned off until the job ends successfully i.e. until the job closes the file normally. Or in case we are editing the file manually in file-aid, the flag gets set when we open the file and gets turned off when we close it. But say the job goes down halfway through the update process or our TSO session expires before we could close the file. The “open-for-output” flag remains turned on. What happens to the file now?

When next time a job/user tries to open the file for output, the file manager trots off to the catalog to turn on the flag. But the flag is already on!! So it guesses (somewhat optimistically) that some other job might have opened the file. It issues a GRS (Global resource serialization) enqueueing on the file to find that out. If the file is not open anywhere else, it comes to know that it was not closed properly during its last use and its time for some catalog cleanup. Enter VERIFY.

Verify is a record management macro like get or put. In our case, the open processing issues an implicit verify against the file. This can be confirmed by the IEC161I type warning messages (RC of 56 and 62) in the sysout. The verify macro will compare the ICF catalog information with the physical VSAM cluster. It starts reading the VSAM dataset CI by CI, starting with the High Used RBA. It compares HURBA value, number of index levels, system time-stamp and many other fields. If the verify is not successful, it will issue a warning message with a return code of 116 (X’74’). Two things worth noting about implicit verify macro:

  1. It will not correct the catalog information. That is the job reserved for IDCAMS verify. It will just issue some warning messages. It is up to us to figure out what to do next. We can continue processing but the data integrity won’t be guaranteed.
  2. Implicit verify is not issued if the file is being opened for input or reset processing. Or if the VSAM file is of type LDS.

IDCAMS verify is an explicit verify command. When an IDCAMS verify is issued against a file, it opens the file, calls the record management verify macro and then closes the file. And it is at the time of closing the file that the ICF catalog gets updated and the “open-for-output” flag gets reset. Two things worth noting about explicit verify macro:

  1. A successful verify does not guarantee that everything is fine now. The catalog statistics may be invalid; the file might have duplicate or missing records. HURBA may be off by a few bytes.
  2. IDCAMS verify does not update the catalog or the VSAM control blocks directly. It relies on the implicit verify and VSAM close processing to do its job.

So whether the verify is successful or not, the structural integrity of a VSAM file is not guaranteed. Its always a good idea to take standard recovery actions on such files.

Manish Kumar

40 responses to “Verifying a VSAM file”

  1. Anonymous

    Hi,
    Do you recomment to Verify the VSAM in the Job before opening the file in OUTPUT or I-O Mode??

    Thanks..
    Chandra

    Reply
  2. Manish Bansal

    It’s always a good idea to issue an exclipit IDCAMS VERIFY on the file before opening it for OUTPUT. And don’t forget to query the RC out of that step.

    Reply
  3. Karthik

    Hi,
    I have question which relates to VSAM file.

    I delete defined a VSAM file(KSDS) and tried to access thru FileAid BROWSE option and its failed saying that “File is empty”.I tried to READ the file using a program too that too found to be not encountering the ENDOF FILE condition.

    Then I inserted a dummy record to file using FIle-Aid in EDIT mode.Then Deleted the DUMMY record just I have inserted to file.

    Now I repeated the first steps I mentioned (using FIle Aid tried to Browse and using the Program tried to READ the file) suprisingly..instead of showing “File is empty ” from FileAid it it opened the contyent window and also the END OF FILE condition occured in Cobol program too…

    Can u tell why its is so?

    Thanks in advance

    Karthik

    Reply
  4. manishbansal

    Hi Karthik,
    When you create a new VSAM file, the HURBA is set to zero. HURBA stands for Highest Used Relative Byte Address. When HURBA is zero, you cannot open the file for input. When you insert a record in the KSDS file and then delete it, the HURBA does NOT get set to zero again. So you are then able to open the file in file-aid.
    However, if the VSAM file was defined with REUSE option, the HURBA does get set to zero after you insert/delete the record.

    Reply
  5. RAMESH

    HOW TO FIND THE LAST RECORD IN KSDS FILE?

    I MEANS TO ASK I WANT TO TRACK THE ONLT LAST RECORD OF KSDS OR ESDS FIEL.

    PLEASE LET ME KNOW THE ANSWER IF YOU KNOW.

    Reply
  6. manishbansal

    Hi Ramesh, can you please tell me some more details? Are you trying to do that through a cobol program? What exactly you want to achieve? And what problem you are facing now?

    Reply
  7. RAMESH

    Hi Manishbansal,

    Nice to see your mail,
    Regarding the question i want to ask you is

    (1) 2 input vsam files files, one is ksds, second one is esds, both files contains the data.

    (2) output file is flat file, should contain the last records from both the input file, this is to perform through cobol program.

    pleas let me know the answer.

    Thanks again,

    Reply
  8. manishbansal

    I still havn’t understood the question properly. Are you trying to merge the two files and then take the last record? Or do you want the last record from each file?
    Anyway, you can get the last record by moving High-Values to the key, doing a Read-Next and then doing a Read-Prev.

    Reply
  9. RAMESH

    Manish, it seems be right answer.

    thank you so much.

    Reply
  10. RAMESH

    What is the use of NOIMBED and NOREPLICATE options in define cluster?

    Please let me know.

    Reply
  11. Biman

    Hi,

    //EW238001 EXEC IDCAMS,SCL=H
    //DD1 DD DSN=FILE1,DISP=SHR
    //DD2 DD DSN=FILE2,DISP=SHR
    //DD3 DD DSN=FILE3,DISP=SHR
    //SYSIN DD *
    VERIFY FILE(DD1)
    VERIFY FILE(DD2)
    VERIFY FILE(DD3)
    /*
    //*
    //***************************************************************
    //*
    //* ABEND WHEN CONDITION CODE OF PREVIOUS STEP GREATER THAN 0
    //*
    //***************************************************************
    //*
    //EW238002 EXEC PGM=SUT302M0,PARM=’0111′,COND=(0,GE,EW238001.IDCAMS)
    //*

    The job abends with condition code 0111. That means that step EW238001 did not execute successfully.

    Can you tell what could be the reason? I read your article on IDACAMS VERIFY function but, still I could not figure out the problem with my job.

    Reply
  12. Nagalakshmi

    What is the use of NOIMBED, NOREPLICATE parameters in Define Cluster, I see the same question has been asked by Ramesh also on Aug 10th. Can you please help me in understanding it.

    Reply
  13. manishbansal

    Replicate:
    A KSDS VSAM cluster has two parts – Data and Index. When we access a VSAM record using a key, the system has to first read the index and get the address of the data record from there. As with any DASD access, the disk platter has to rotate before the desired index entry comes under the magnetic head. This time lag is called rotational delay and is typically measured in milliseconds. In the days gone by, the hard disks used to be very slow and rotational delay could be as much as 10-15 milliseconds. To minimize this delay, each index entry is replicated or repeated as many times as would fit on a track. This insures that no matter where the magnetic head is on a platter, the disk has to spin very little to read the desired index.

    Imbed:
    The other type of delay that occurs during a hard disk read is called seek time. This is the time taken for the magnetic head to position itself over the desired track. To minimize this delay, the sequence set portion of the index is replicated on to the first track of the data CA (Imbed). And each sequence set index points to the next data CA and so on. This minimizes seek time only in case of sequential access. Once the magnetic head has been positioned over the first tack and sequence set retrieved, the head assembly does not have to move again. Only the disk has to spin to read the next record and so on. This option does not improve performance in case of random access.

    Note:
    Both these options are no longer supported because the hard disks nowadays are fast enough to not warrant such measures. Even if you do specify them while allocating the VSAM cluster, they would be ignored.

    Reply
  14. Nagalakshmi

    Thank You So Much Manish. I need to remove these options in the JCL and the programs. Beforing removing it, I wanted to know the use of it.

    Reply
  15. Swapnali

    Hi,

    How to set the HURBA value of the VSAM file in TSO and through the batch job?

    I have one VSAM KSDS file whose records RBA is not in a sequence. Record length of the file is 1000 bytes. So in this case the RBA of the first record should be from 0 to 999, then for the second record it should be 1000 – 1999 and so on. But it is not that way. RBA values are not in sequence. Due to this I am not able to access the record sequentially.

    Let me know what could be the reason behind this?

    Regards,
    Swapnali

    Reply
  16. Manish Bansal

    Hi Swapnali, I am not able to understand your question. If you have a VSAM KSDS file, what is the role of RBA there? You should be accessing this file by its key.
    Regarding HURBA, you don’t have to set it manually. It gets set automatically when you write a record to it.

    Reply
  17. Swapnali

    Hi Manish,

    Thanks for your reply.

    Yes, I can access the VSAM KSDS file by its key. But the requirement was to access the VSAM KSDS file sequentially and the file which I was trying to access, has the RBA out of sequence.

    Right now I have created one more file of the same structure and copied all the records from the affected VSAM file to the new file, by using REPRO. New file has the RBA in sequence.

    I just want to know that why RBA got out of sequence in OLD file?

    Reply
  18. Manish Bansal

    Hi Swapnali, I do not know the exact reason for this but it typically happens when a job abends while the file update was in progress. Or if you don’t close the file at the end of the program, the system will try to close it for you but sometimes it fails and the file gets corrupted. Try doing an IDCAMS VERIFY on this and see what comes up.

    Reply
  19. jagadesh

    i want to read a ksds file from the last record because to update the same file i need the count of the file which is written one day before

    Reply
  20. Anurag Johri

    In my Job a step does a REPRO for a VSAM file but when VSAM file is empty it abends with RC=12. I think of a add a IDCAMS VERIFY before that step. So if the file is empty i can skip the main step and job runs fine..But i dont know what are the RC of IDCAMS VERIFY when file is empty or there is data in VSAM file.

    Reply
  21. Sridhar

    Hi Manish,

    Can you please advise, how to read a vsam file using partial key.

    Reply
  22. Jasmine Kar

    Hi,

    Can anyone explain me what is the meaning of SCL parameter in the IDCAMS step. I have never seen this before. Here is an example:

    //XW931003 EXEC IDCAMS,SCL=’*’
    //SYSIN DD *
    DELETE PCC1.XW9310C0.NDM
    DELETE PCC1.XW9311C0.NDM

    Thanks,
    Jasmine

    Reply
  23. Sudip

    File1 – VSAM
    File2 – VSAM
    File3 – Flat
    What i have to do is,
    Read flat file Search that record in VSAM files If found then update Flat file with one field in VSAM file.

    Reply
  24. Sudip

    can u help me ……… i need a cobol programme

    Reply
  25. maridurai

    hi
    i having error in idcams which showing return code 16 while running the jcl,

    Reply
  26. anu

    hi, can i have the code to find the last record in a vsam file?

    Reply
  27. SELVAPRIYA

    how can i see the last record in vsam file while i am in reading the records?

    Reply
  28. Appu

    How to identify a record in KSDS file using the partial key??

    Reply
  29. eula

    Working with Newly created VSAM.
    U’ll get RC 35 if u try to read/write, or RC 12 for verifying newly created VSAM through batch job.
    The trick is to open, add 1 record and delete it manually.

    at my place, we use KSDSINIT program (which do that trick using assembler or something – i dont who the creator is, i doubt that it is provide by the system since i found nothing when i googling KSDSINIT keyword )

    //KSDSINIT EXEC PGM=KSDSINIT,COND=(0,NE)
    //STEPLIB DD DSN=SYS1.LINKLIB,DISP=SHR
    //KSDS DD DSN=MY.DSN,DISP=OLD
    //SYSPRINT DD SYSOUT=*

    or you can use second alternative, put low-values after you define VSAM. But of course, your must ensure to ignore that low-values in your program.

    //BSLSREP EXEC PGM=IDCAMS,COND=(0,NE)
    //SYSPRINT DD SYSOUT=*
    DELETE INTLX.BSS.BSLSREP PURGE CLUSTER
    IF MAXCC = 8 THEN SET MAXCC = 0
    DEFINE CLUSTER –
    (NAME(MY.DSN) –
    INDEXED –
    SHAREOPTIONS(2,3) –
    NOERASE –
    RECOVERY –
    NOWRITECHECK –
    NONSPANNED) –
    DATA –
    (NAME(MY.DSN.DATA) –
    KEYS(15 0) –
    CONTROLINTERVALSIZE(4096) –
    CYLINDER(2 1) –
    FREESPACE(0 0) –
    RECORDSIZE(30 30) –
    –
    ) –
    INDEX –
    (NAME(MY.DSN.INDEX) –
    CONTROLINTERVALSIZE(4096) –
    CYLINDER(1 1) –
    NOIMBED –
    UNORDERED –
    NOREPLICATE)
    /*
    //**——
    //WRITORMS EXEC PGM=DITTO,PARM=’JOBSTREAM’,COND=(0,NE)
    //SYSPRINT DD SYSOUT=A
    //OUTFILE DD DSN=MY.DSN,DISP=SHR
    //SYSIN DD *
    $$DITTO BV OUTPUT=OUTFILE,RECSIZE=30,NLRECS=1,FILLCHAR=X’00’
    /*

    in my batch program:
    READ MYDSN
    AT END MOVE ‘Y’ TO WS-SWITCH-END-OF-FILE.

    IF MYDSN-KEY = LOW-VALUE
    READ MYDSN
    AT END MOVE ‘Y’ TO WS-SWITCH-END-OF-FILE.

    Reply
  30. eula

    i forget to say thank you to manishbansal for the article (this one and directory blocks).
    keep on sharing 😀

    Reply
  31. Anthony

    Hi,

    Can you tell me if there is a way to verify the internal control structures of a VSAM file. Before, we used a tool called VSAM Mechanic, which did an outstanding job fixing the broken RBAs, etc. Is there a facility in IDCAMS that can do this, as we may no longer have this tool in shop?

    Regards,
    Anthony

    Reply
    1. Manish Bansal

      IDCAMS VERIFY command should be able to do it. I don’t think it can repair any corruption but it can certainly verify the file integrity.

      Reply
  32. surya

    Hi Manish,
    I am using a COBOL program to open a VSAM file in I-O mode. It fails with status 90. I recreated it with shareoptions 3,3. Still does not seem to make any difference. I am pretty unsure of how to proceed from here. Any thoughts will be welcome.
    Thanks much.

    Reply
    1. Manish Bansal

      Hi Surya,
      Sorry for the late reply. It would be tough for me to answer this without knowing more details about the job setup etc.

      Regards,
      Manish.

      Reply
  33. Ivan H

    Hi Manish,
    Can i verify a vsam file which is open as read only in CICS?

    Reply
    1. Manish Bansal

      @Ivan,
      I am not 100% sure but it should be possible as the VERIFY command works at the system level whereas the read-only mode is valid only for the program which is using that file. In other words, the VSAM file itself has no read-only or read-write mode at the system level.

      Reply
      1. Ivan H

        Thanks Manish,
        To elaborate the scenario – I have a file “A” which is open in CICS with the FCT set as “READ ONLY”. I try to execute a TSO / IDCAMS VERIFY on this file and receive an error message “Dataset already in Use, try later”. This message would make more sense if this file was updatable in CICS. Any clue on this one?

        Reply
  34. Anonymous

    how read ksds file dynamically and write after 10 records from ksds file to flat file.?

    Reply
  35. ankoo

    Thank you Manish Bansal for the good article

    Reply
  36. Facebook Web Development

    Greetings I am so grateful I found yohr website,
    I really found you by error, while I was searching on Bing for something else, Anyways I am here
    now and would just like to say many thanks for a fantastic post and a
    all round thrilling blog (I also love the theme/design), I don’t have time to look
    over it aall at the moment but I have saved it and also added
    in your RSS feeds, so when I have time I will be back to read much more, Please do keep up
    the great b.

    Reply

Leave a reply to Facebook Web Development Cancel reply