Hi Rob and Joe, I forgot to tell you that you can of course make one list file to put all merge requests in, namely all of below from mergeTop to mergeBTAGGING can be put in one file so that you submit one task (one list file) for all. Nurcan. On Jul 13, 2012, at 12:58 PM, Nurcan Ozturk wrote: > Hi Rob and Joe, > > Here are the updated template list files for merge tasks, from p1131 to p1141: > > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000001.group_MERGE_list.recon.LIST.p1141.mergeTop > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000004.group_MERGE_list.recon.LIST.p1141.mergeSUSY > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000005.group_MERGE_list.recon.LIST.p1141.mergeJETMET > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000006.group_MERGE_list.recon.LIST.p1141.mergeEGAMMA > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000007.group_MERGE_list.recon.LIST.p1141.mergePHOTON > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000008.group_MERGE_list.recon.LIST.p1141.mergeSMWZ > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000009.group_MERGE_list.recon.LIST.p1141.mergeHSG2 > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000010.group_MERGE_list.recon.LIST.p1141.mergeHSG5 > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000011.group_MERGE_list.recon.LIST.p1141.mergeEXOTICS > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000012.group_MERGE_list.recon.LIST.p1141.mergeSMQCD > http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000013.group_MERGE_list.recon.LIST.p1141.mergeBTAGGING > > Joe, could you please monitor the merge tasks that will be submitted by Rob today, from the task query page: > > http://panda.cern.ch/server/pandamon/query?mode=taskquery > > by choosing "p1131" and "gr_*". Hope all tasks work fine with p1141. > > Joe, please document on our wiki: > > https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/DPDProductionTaskSubmissionAndMonitoring > > what we have discussed on open-ended requests and their merge task submissions as well as their monitoring as above. > > I encourage both of you being proactive in documenting, the format is not so important on wiki, we can always revise later, > just a simple text is fine now. > > Thanks, > Nurcan. > > > > On Jul 12, 2012, at 11:48 AM, Nurcan Ozturk wrote: > >> Hi Rob and Joe, >> >> As you know I have started open-ended requests (starting from first run after technical stop - 00206299) for all groups last Friday. >> I have submitted majority of merge tasks on them, will do the rest today. >> >> I would like you to carefully check the list files I made for both AOD->NTUP and NTUP->NTUP (merge) submissions, listed here: >> >> 1. AOD->DAOD: >> >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000001.group_MERGE_list.recon.LIST.p1130 >> >> 2. AOD/ESD/DESD->NTUP >> >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000021.group_MERGE_list.recon.LIST.p1130 >> >> 3. Merge tasks (I added a string (for NTUP type) at the end only for your reference): >> >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000001.group_MERGE_list.recon.LIST.p1131.mergeTop >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000004.group_MERGE_list.recon.LIST.p1131.mergeSUSY >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000005.group_MERGE_list.recon.LIST.p1131.mergeJETMET >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000006.group_MERGE_list.recon.LIST.p1131.mergeEGAMMA >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000007.group_MERGE_list.recon.LIST.p1131.mergePHOTON >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000008.group_MERGE_list.recon.LIST.p1131.mergeSMWZ >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000009.group_MERGE_list.recon.LIST.p1131.mergeHSG2 >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000010.group_MERGE_list.recon.LIST.p1131.mergeHSG5 >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000011.group_MERGE_list.recon.LIST.p1131.mergeEXOTICS >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000012.group_MERGE_list.recon.LIST.p1131.mergeSMQCD >> http://nozturk.web.cern.ch/nozturk/DPDProduction/open-ended/data12_8TeV.000013.group_MERGE_list.recon.LIST.p1131.mergeBTAGGING >> >> I will take care of 1) and 2) for now. And I like to ask you to take care of 3) from now on. >> >> We have total 24 submissions in 1) and 2), so 24 submissions for merge tasks (to be checked with Junji). >> 11 of the merge tasks I submitted yesterday as in 3). The rest I will do today. Eventually you two will be >> taking care of all merge task submissions on open-ended requests and this needs to be done once a day >> as our target delivery time is 48 hours after a new run/dataset is available to process. >> >> Rob should have some ideas on merge task submissions now. The main point is to find which datasets are newly produced >> from the list of datasets already in 3). This is tedious work at the beginning but becomes pretty routine after you do it few times. >> Panda monitor just list everything: >> >> - start from: http://panda.cern.ch/server/pandamon/query?mode=taskquery >> - from the list file in 2),put requesters email to "Submitted by", p-tag to "Configuration tag", >> group name to "Group" and a string for "Input dataset" into the task query page, examples are: >> - data12_8TeV%physics_Egamma.merge.AOD.f% >> - data12_8TeV%physics_Egamma.merge.DESD_ZEE% >> - data12_8TeV%debugrec_%merge.AOD.g% >> - data12_8TeV%physics_Egamma.recon.DESD_PHOJET.f% >> - data12_8TeV%physics_JetTauEtmiss.merge.DESD_CALJET.f% >> - data12_8TeV%physics_Muons.recon.DESD_ZMUMU.f% >> >> Let's talk on Skype on this as I like you to submit the new merge tasks first thing on Friday morning. Rob can then submit all 11 >> tasks Friday morning and then share the work in the next round by splitting the all merge tasks between Rob and Joe. >> >> How about 10am CERN time on Skype? >> >> Cheers, >> Nurcan. >> >> >> >> >> >> >> >> >> >> >> >>