Releases and Patching: Difference between revisions

From OpenPetra Wiki
Jump to navigation Jump to search
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Releases ==
== Releases ==
The deliverables are:  
The deliverables are:  
* the tar.gz files (server-debian-postgresql, server-debian-centos, client) which could be the base for manual installs, and for RPMS
* the tar.gz binary file which could be the base for manual installs, and for RPMS. see https://get.openpetra.org/latest
* the standalone installer for Windows
* the remote client installer for Windows (not published on Sourceforge, since it would be bound to an IP address)
** a special tar.gz file that is used as a base for building the patches (see below)


=== Things todo for an Open Source release ===
=== Things todo for an Open Source release ===
Line 11: Line 8:
* develop on dev branch.
* develop on dev branch.
* Test with Appveyor (https://ci.openpetra.org), Test mysql on LBS (https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetra-test-server-mysql).
* Test with Appveyor (https://ci.openpetra.org), Test mysql on LBS (https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetra-test-server-mysql).
* update Release Notes in https://github.com/openpetra/openpetra/tree/dev/setup/petra0300/releasenotes
* update Release Notes in https://github.com/openpetra/openpetra/tree/dev/setup/petra0300/releasenotes; see python script get_issues.py https://gist.github.com/tpokorra/d608a8dde183e7bb51f135e052b301cc
* merge into test branch.
* merge into test branch.
* update Weblate and Sync translations to/from Github: https://weblate.openpetra.org/
* build the binary tarball on lbs (https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetra-bin-tarball)
* build the binary tarball on lbs (https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetra-bin-tarball)
* update http://get.openpetra.org/openpetra-latest-bin.tar.gz to point to the new tarball
* done automatically: update https://get.openpetra.org/openpetra-latest-bin.tar.gz to point to the new tarball
* build openpetranow-mysql-test on lbs (https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetranow-mysql-test)
* install on demo.test.openpetra.org (similar to demo.openpetra.org)
* install on demo.test.openpetra.org (similar to demo.openpetra.org)
* try to run /home/openpetra/openpetra-server.sh update
* try to run /home/openpetra/openpetra-server.sh update
* build the latest demo databases: https://lbs.solidcharity.com/package/solidcharity/openpetra/demodb
* then again build the binary tarball so that the latest base.yml.gz is included, and rebuild openpetranow-mysql-test
* download the latest demo database to the demo.openpetra.org container
* try restore demo database from command line. try running month end. try a report. try to edit partner
* try restore demo database from command line. try running month end. try a report. try to edit partner
* install on demo.openpetra.org
* install on demo.openpetra.org
Line 34: Line 34:
* increase the version in db/version.txt to 2016.07.0-0
* increase the version in db/version.txt to 2016.07.0-0
* build the test branch of the packages:
* build the test branch of the packages:
** https://lbs.solidcharity.com/package/tpokorra/openpetra/openpetranow-demo#test_centos/7/amd64
** https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetra-bin-tarball
** https://lbs.solidcharity.com/package/tpokorra/openpetra/openpetranow-test#test_centos/7/amd64
** https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetranow-mysql-test
** https://lbs.solidcharity.com/package/tpokorra/openpetra/openpetranow-standalone-test


* [[Manual tests before a release]]
* [[Manual tests before a release]]


===== [[Releases and Patching#Mantis Tracker|Mantis Tracker]]=====
* All steps under this section can take some time and can be done a few days in advance to get the most work involved done.
** Step [[Releases and Patching#Creating Static 'Bugs Resolved in this Release' HTML File|Creating Static 'Bugs Resolved in this Release' HTML File]]
*** Doing this step a few days before the release has the advantage that it is easier for the developers to see what has been achieved by them between the last and the the upcoming release, which in turn helps in writing the Release Notes and the News Update. (That step makes most sense if the steps before this step are also done already as the produced bug list will then be as complete as possible at the time.)
* Obviously all the steps also need to be performed on the day of the release to ensure that everything is covered.
===== Preparing the text for the News Update=====
* Compiling the text for the News Update is a time-consuming process as the most important completed tasks of each developer should find their way into them and therefore the developers need to be asked for that information and once it comes back it needs to be compiled and formatted. Usually this can all be done once the testing phase begins which precedes the release - that might even be a week before the release because even if there is still a major new functionality that might get added before the release will be done this will be known by now and can be written up already.
** Obviously the text needs to be looked at carefully at the day of the release, but most of time-consuming work will have been done by then.
===== Preparing the text for the [[Releases and Patching#Release Notes|Release Notes]]=====
===== Preparing the text for the [[Releases and Patching#Release Notes|Release Notes]]=====
* Usually this can be done once the testing phase begins which precedes the release - that might even be a week before the release because even if there is still a major new functionality that might get added before the release will be done this will be known by now and can be written up already.  
* Usually this can be done once the testing phase begins which precedes the release - that might even be a week before the release because even if there is still a major new functionality that might get added before the release will be done this will be known by now and can be written up already.  
** Obviously the text needs to be looked at carefully at the day of the release, but most of time-consuming work will have been done by then.  
** Obviously the text needs to be looked at carefully at the day of the release, but most of time-consuming work will have been done by then.  
*** Don't commit the text yet to the bazaar repository, but keep it in a separate file so that you can copy the contents of that file into the file that gets committed to the bazaar repository on the day of the release.
=====Preparing up-to-date screenshots=====
=====Preparing up-to-date screenshots=====
* Check which (if any) screenshots on the 'Screenshots' page of the OpenPetra website need updating.  
* Check which (if any) screenshots on the 'Screenshots' page of the OpenPetra website need updating.  
Line 58: Line 48:
** Screenshots are listed on that page in a smaller size, and when one clicks on them the full size is shown. The smaller size can also already be prepared in advance.
** Screenshots are listed on that page in a smaller size, and when one clicks on them the full size is shown. The smaller size can also already be prepared in advance.


==== Update translations from/to Launchpad ====
==== Update translations from/to Weblate ====
* Download the latest language files from Launchpad using <code>nant translation</code>
* Download the latest language files from Weblate. Do a synchronization, and that might create a pull request.
** Adding new languages (if desired)
*** New languages might have been downloaded after running <code>nant translation</code>. These would show up in the 'Unversioned' section of Bazaar Explorer. If we really want to add one of those new languages to OpenPetra then perform these steps:
**** Add DLL in section '<code>[Files]</code>' in file <code>setup\petra0300\standalone\standalone.iss</code>;
***** e.g. <code>Source: ..\..\..\delivery\bin\es-ES\OpenPetra.resources.dll; DestDir: {app}/bin30/es-ES</code> for Spanish
**** Add language at task '<code>downloadTranslations</code>' in file <code>i18n\i18n.build</code>;
***** e.g. <code><move file="${dir.root}/i18n/de.po" tofile="${dir.root}/i18n/de-DE.po"/></code> for German
**** Bazaar-Add the language file that is showing in the 'Unversioned' bazaar section so it is under Version Control.
* Generate the latest translation template
** run <code>nant buildTranslationTemplate</code>
*** after running this, check any modified source code files for additions of 'Catalog.GetString' and if they make sense, i.e. they are not added for placeholder text of Labels, TextBoxes or Form Titles, etc - or codes that must not be translated.
*** also, look at a diff of the file <code>template.pot</code> to see if any additions make sense, i.e. they are not added for placeholder text of Labels, TextBoxes or Form Titles, etc - or codes that must not be translated. ''Warning: This can be a time-consuming process!'' (Depending on how much time has been between the last release and now.)
** if you have found translations that do not make sense,
*** add them to the file <code>\trunk\i18n\donottranslate.po</code> in the pattern that can clearly be seen in this file;
*** re-run <code>nant buildTranslationTemplate</code> to have the translations that got added to the file <code>\trunk\i18n\donottranslate.po</code> removed from the file <code>template.pot</code>.
* run <code>nant uncrustify</code> since <code>nant translation</code> reformats some of the code and the changed source code files might not conform to our uncrustify rules.
* run <code>nant quickClean</code> and <code>nant generateSolution</code> to ensure the whole source still compiles
** automatic additions of 'Catalog.GetString' to source code files could have happened where there was no 'using Ict.Common;' in the file; those files will not compile and will need to have that using statement added manually. (See [https://tracker.openpetra.org/view.php?id=1889 Bug #1889])
* Commit to Launchpad trunk using Bazaar.
 
==== Old: Mantis Tracker ====
===== Ensuring that all completed Bugs are set to resolved =====
Recent changes to our bug work-flow introduced a new bug status, 'completed'. Setting a bug to 'completed' means that the developer has fixed the bug in his own branch, but that it is not yet merged into trunk.
 
Before we finalise the the list of bugs that is fixed in this release we should therefore look into Mantis to find out if there are any bugs left in 'completed' state that indeed made it into trunk by the time of the release. To do this, follow the steps outlined here.
 
* Using Mantis,
** filter for all bugs with 'completed' state and 'Hide Status' equals 'None'.
** Open Bazaar Explorer on trunk and open the Log.
** In the Log screen set the Search to 'Bugs' and enter one bug number after the other that you see on the filtered Mantis output.
*** if no trunk revision is found that contains this bug, try switching the search to 'Messages' and try again. If still no trunk revision is found, contact the developer to find out in which trunk commit the bug 'slipped' into trunk, of if it is indeed not in trunk yet.
*** if you could determine a trunk revision that fixes this bug, edit the bug and resolve it, setting the Commit Link to the revision where it went into trunk and add as a Note 'Resolved by #developer name#'. Set the 'Fixed in' to this release number (or to the corresponding 'Dev' release number if it was an internal bug).
** Do this for all bugs on the filtered output until you are sure that either there are no bugs left with 'completed' status or that the ones that are left are indeed not in trunk and hence not part of this release.
 
===== Ensuring that all fixed Bugs have set "Fixed in Version" =====
We need to make sure that every developer has set "Fixed in Version" on all the bugs that have 'Closed' or 'Resolved' state. As this is normally forgotten on quite a few bugs, it must be done manually before a release in order to have a complete list of bugs that were fixed in this release.
 
* Using Mantis,
** filter for all bugs with 'resolved' state that have 'Fixed in Version' equals 'none', 'Resolution' equals 'fixed' and 'Hide Status' equals 'None'.
** do the same for 'closed' state (e.g. in a new Tab of your web browser).
** In both steps, make a note of all bug numbers that
*** have been 'closed'/'resolved' ''since the date of the last release''.
*** are ''not 'internal' bugs'' (which should be set to a 'developer release number' instead of a release number of an 'official release'!).
**** To be able to determine that open each bug and class it either as one that should get included in the bug list of the 'official release' or as one that should get a 'developer release number'.
*** this is best done as follows
**** if there are few bugs to be handled
***** create an empty text document, e.g. in Notepad
***** copy each mantis bug number off its web page and paste it into that text document, putting commas between the bug numbers.
***** once finished that whole text line can be copied and pasted on the command line of the MantisUpdateVersions Tool (see next bullet point).
**** if there are many bugs to be handled
***** create an empty text document, e.g. in Notepad
***** click on 'Excel Export', open the downloaded Excel Document, select the content of the whole first column excluding the title row and copy it to the clipboard.
***** paste into the empty text document that you created in the previous step
***** remove any bugs that should not get processed e.g. because they were fixed/closed before the last release or because they are 'internal bugs'
***** replace all CR+LF's with commas
***** once finished that whole text line can be copied and pasted on the command line of the MantisUpdateVersions Tool (see next bullet point).
* Run the MantisUpdateVersions Tool
** The tool resides in <code>\setup\tools\MantisUpdateVersions</code>. As it isn't get built by <code>nant generateSolution</code> you need to load the C# Project 'MantisUpdateVersions.csproj' into your favourite IDE and compile it.
** After the compilation the DLL's Ict.Common.dll and 'Futureware.MantisConnect.dll' need to be copied to <code>'\setup\tools\MantisUpdateVersions\obj\x86\Debug'</code>.
** Open a command shell and change the directory to <code>'\setup\tools\MantisUpdateVersions\obj\x86\Debug'</code>
** Execute the following command: <code>MantisUpdateVersions.exe -sf-username:pokorra -sf-pwd:xyz -bug-id:abc,def,ghi -version-fixed-in:"Alpha 0.2.20"</code>, replacing the value for <code>-sf-username</code> and <code>-sf-pwd</code> with your MantisBT username and password, the string that follows after '-version-fixed-in:' with the current release number and <code>abc,def,ghi</code> with the 1..n Mantis Bug ID's that you found in the two Mantis filter steps.
*** ''NOTE'': The MantisBT user that is specified in this command needs to have 'administrator' privileges in the Mantis Bugtracker so that the commands that are issued by this program can succeed!
 
===== Creating Static 'Bugs Resolved in this Release' HTML File =====
We need to create a static HTML page that contains the list of all bugs that got resolved/closed in this release. This page gets uploaded to SourceForge and is linked to from the Release Notes and News Updates.
* Simply filtering Mantis and creating a permalink [using the 'Create Permalink' functionality on the search result page] does not work with Mantis in the version that is used on Sourceforge!
* Instead, do the following:
** In FireFox (Chrome is bad with external links),
*** go to https://tracker.openpetra.org/view_all_bug_page.php
*** First reset any Filter that you might have created (by selecting '[Reset Filter]' and clicking 'Use Filter'); then click on 'Advanced Filters', then create the following Filter:
**** Click on 'Fixed in version' and select the current release number
**** Click on 'Status': Click on 'resolved', press <SHIFT> and click on 'closed' ('resolved', 'tested', 'approved' and 'closed' should now all be highlighed)
*** Click 'Apply Filter'.
*** Click on the 'Print reports' link and save the file to some local directory, plus the Images, CSS, JavaScript and Favicon in a folder named after the html file (if the file is named e.g. 'BugsFixed_Alpha-0.2.8.htm' then the folder needs to be named 'BugsFixed_Alpha-0.2.8_files')
* Upload that file and folder using some FTP program (directory <code>username/openpetraorg@web.sourceforge.net:/home/project-web/openpetraorg/htdocs/ReleaseDetails</code>).
* Note: To obtain an Excel File instead of a static HTML page, simply click 'Excel Export' instead of 'Print reports'!


==== Release Notes ====
==== Release Notes ====
Line 148: Line 63:
* add new patch .cs and .sql file to https://github.com/openpetra/openpetra/tree/master/csharp/ICT/Petra/Server/lib/MSysMan/DBUpgrades
* add new patch .cs and .sql file to https://github.com/openpetra/openpetra/tree/master/csharp/ICT/Petra/Server/lib/MSysMan/DBUpgrades
* modify file <code>version.txt</code> in directory db/patches so it contains the Version Number of the OpenPetra release that you are creating.
* modify file <code>version.txt</code> in directory db/patches so it contains the Version Number of the OpenPetra release that you are creating.
** Alternative: use the commandline argument <code>-D:ReleaseID=0.0.2.0</code> [use current release number] to specify the build number when running <code>nant buildWindowsStandalone</code> in step [[Releases and Patching#Installers | Installers]].


====Final Steps====
====Final Steps====
===== Old: Bazaar =====
===== Building the release =====
* Tag trunk with the name of the new release (e.g. <code>ALPHA-0.2.22</code> [use current release number])
 
===== Old: Installers =====
* On a Windows machine:
** Perform a <code>bzr update</code> on your check-out of Bazaar trunk
** Run <code>nant buildWindowsStandalone -D:ReleaseID=0.0.2.0</code> [use current release number]
** Perform a <code>bzr revert</code> to get rid of i18n files that are not wanted.
 
* On a linux machine (on openpetrabuild: /home/timop/bzr):
** Perform a <code>bzr update</code> on your check-out of Bazaar trunk
** Run <code>nant buildLinuxSourceforgeRelease -D:ReleaseID=0.0.2.0</code> [use current release number]. 
** Perform a <code>bzr revert</code> to get rid of i18n files that are not wanted.
 
* Installers end up in the <code>/delivery/</code> directory.
* Test installers on Windows and Linux.
* Upload all installers to SourceForge using some FTP program: <code>username@frs.sourceforge.net:/home/pfs/project/o/op/openpetraorg/openpetraorg</code>
 
===== New: Building the release =====
* increase the version in db/version.txt
* increase the version in db/version.txt
* merge branch into master, and into test and prod branches
* merge branch into master, and into test and prod branches
Line 175: Line 71:
** https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetranow-mysql-test
** https://lbs.solidcharity.com/package/solidcharity/openpetra/openpetranow-mysql-test
* create a tag and a release in https://github.com/openpetra/openpetra
* create a tag and a release in https://github.com/openpetra/openpetra
** upload the rpm, eg. https://lbs.solidcharity.com/repos/solidcharity/openpetra/centos/7/x86_64/openpetranow-mysql-test-2019.06.0-3.x86_64.rpm


===== Making the Release Known to the Public =====
===== Making the Release Known to the Public =====
Line 199: Line 94:
** Send out information on mailing lists (English, German).
** Send out information on mailing lists (English, German).
*** Content of the emails: exactly the same as in the tweets - except if there is a News Update that corresponds with the release: in this case the content of what got posted in the 'News' section of the website.
*** Content of the emails: exactly the same as in the tweets - except if there is a News Update that corresponds with the release: in this case the content of what got posted in the 'News' section of the website.
===== Mantis Tracker =====
* Next Release needs to be added to all projects
** As this is tedious if done manually use the tool <code>MantisUpdateVersions.exe -sf-username:pokorra -sf-pwd:*** -release-version:0.6.10.0</code> from <code>/setup/tools/</code> for that [use ''current'' release number!]
*** In case you haven't yet built the MantisUpdateVersions tool: go to [[Releases and Patching#Ensuring_that_all_fixed_Bugs_have_set_.22Fixed_in_Version.22 |this section]] and drop down to the bullet point 'Run the MantisUpdateVersions Tool'. Follow the instructions there to get the tool built and to open the command shell in the right folder.
*** The last dot and digit in the '-release-version:' string are required, but are getting ignored (as of March 2016).
** For an unknown reason the adding of the Next Release to 'all projects' doesn't add it to the 'Tools' Project; thus the Project Versions that the tool adds to all other Projects need to be manually added to this Project.
=====DB Patches for Developers' Version=====
* add new 'empty' patch sql file to db/patches
** this needs to have the new ''developers' '' version number, which is always an uneven number that is one number higher than the release that was just done.
*** Example: if the current release that was just done was 0.2.24, add an empty file by the name of patch_0.2.24_0.2.''25''.sql
* modify file <code>version.txt</code> in directory db/patches so it contains the Version Number of the new ''developers' '' version number (e.g. 0.2.''25''-0).


== Patches ==
== Patches ==
Line 220: Line 102:
=== Testing the server upgrade ===
=== Testing the server upgrade ===
on a testing machine: test.openpetra.org
on a testing machine: test.openpetra.org
* Install the previous version:
* Install the previous version on a clean container:
  curl https://get.openpetra.org | bash -s old
  curl https://get.openpetra.org | bash -s old
* run the upgrade:
* run the upgrade:
  /home/openpetra/openpetra-server.sh update
  /home/openpetra/openpetra-server.sh update
* Test the login on https://test.openpetra.org
* Test the login on https://test.openpetra.org
=== Reverting to previous version ===
export OLDVERSION="2020.04.0-3"
export TagDemoDB="UsedForNUnitTests-202004"
export OP_CUSTOMER=op_demo
yum remove openpetranow-mysql-test
yum install openpetranow-mysql-test-$OLDVERSION.x86_64 --enablerepo=lbs-solidcharity-openpetra
MYSQL_CMD="drop database $OP_CUSTOMER" /home/openpetra/openpetra-server.sh mysql
/home/openpetra/openpetra-server.sh initdb
demodbfile=/tmp/demoWith1ledger.yml.gz
curl --silent --location https://github.com/openpetra/demo-databases/raw/$TagDemoDB/demoWith1ledger.yml.gz > $demodbfile
/home/openpetra/openpetra-server.sh loadYmlGz $demodbfile
Now update:
yum clean expire-cache --disablerepo="*" --enablerepo="lbs-solidcharity-openpetra"
/home/openpetra/openpetra-server.sh update

Latest revision as of 04:25, 27 October 2021

Releases

The deliverables are:

Things todo for an Open Source release

Short version

What can/should be done a day or even a few days before the actual release date

Performing all the steps that are listed here on this page can take - on average - a day if the last release has been some months ago. However, the time it takes on the day of the release can potentially be reduced by doing several steps a day or even a few days in advance.

These are the steps that can be done before the actual release:

Testing the release packages before the release
Preparing the text for the Release Notes
  • Usually this can be done once the testing phase begins which precedes the release - that might even be a week before the release because even if there is still a major new functionality that might get added before the release will be done this will be known by now and can be written up already.
    • Obviously the text needs to be looked at carefully at the day of the release, but most of time-consuming work will have been done by then.
Preparing up-to-date screenshots
  • Check which (if any) screenshots on the 'Screenshots' page of the OpenPetra website need updating.
  • If any need updating and these screens won't change any more until the release is done, take those screenshots now and store them until the day of the release has come as this process takes some time.
    • If any screenshot is to be replaced check the file name of the existing screenshot and name the new one accordingly.
    • Screenshots are listed on that page in a smaller size, and when one clicks on them the full size is shown. The smaller size can also already be prepared in advance.

Update translations from/to Weblate

  • Download the latest language files from Weblate. Do a synchronization, and that might create a pull request.

Release Notes

  • Update Release notes files in setup/petra0300/releasenotes. (English: releasenotes.html, German: releasenotes_de.html)
    • Format:
      • never empty the contents of that file as it is meant to be cumulative!
      • rather, add the Release Notes that pertain to the current release on top of the file.
      • follow the format of the previous Release Notes.
  • Commit the changed files to Launchpad trunk using Bazaar.


DB Patches

Final Steps

Building the release
Making the Release Known to the Public
Website
  • Screenshots
    • see above for the preparation of up-to-date screenshots
    • update the 'Screenshots' page of the website (English, German).
  • Release Notes
    • Post Release Notes on the 'Download' page of the website (English, German).
      • Format of the Release Notes: refer to previous posts.
      • Include download link for the Windows Installer and the Linux packages that pertain to this release.
      • Include link to the static HTML file that contains the list of closed bugs that pertains to this release (created in an earlier step in these instructions).
  • News Update
    • If there is a News Update that corresponds with the release (which often is the case): post news update in the 'News' section of the website (English, German).
      • Format of the News Update: refer to previous posts.
      • Include link to the 'Download' page of the website (English, German).
Notifications
  • Send out a 'Tweet' on Twitter
    • Tweet about the release on twitter (English [user: openpetraorg], German [user: openpetraorgde])
    • Format of the Tweet: refer to previous tweets that pertain to releases.
  • Mailing Lists
    • Send out information on mailing lists (English, German).
      • Content of the emails: exactly the same as in the tweets - except if there is a News Update that corresponds with the release: in this case the content of what got posted in the 'News' section of the website.

Patches

patching the database

OP_CUSTOMER=op_dev /home/openpetra/openpetra-server.sh upgradedb

Testing the server upgrade

on a testing machine: test.openpetra.org

  • Install the previous version on a clean container:
curl https://get.openpetra.org | bash -s old
  • run the upgrade:
/home/openpetra/openpetra-server.sh update

Reverting to previous version

export OLDVERSION="2020.04.0-3"
export TagDemoDB="UsedForNUnitTests-202004"
export OP_CUSTOMER=op_demo
yum remove openpetranow-mysql-test
yum install openpetranow-mysql-test-$OLDVERSION.x86_64 --enablerepo=lbs-solidcharity-openpetra
MYSQL_CMD="drop database $OP_CUSTOMER" /home/openpetra/openpetra-server.sh mysql
/home/openpetra/openpetra-server.sh initdb
demodbfile=/tmp/demoWith1ledger.yml.gz
curl --silent --location https://github.com/openpetra/demo-databases/raw/$TagDemoDB/demoWith1ledger.yml.gz > $demodbfile
/home/openpetra/openpetra-server.sh loadYmlGz $demodbfile

Now update: 

yum clean expire-cache --disablerepo="*" --enablerepo="lbs-solidcharity-openpetra"
/home/openpetra/openpetra-server.sh update
Retrieved from ‘https://wiki.openpetra.org/index.php?title=Releases_and_Patching&oldid=4887’