danielle
Pivian
Sieres
2
May '18
Sieres Reporting in, with the next round of QA Proposals,
This proposal is to continue the QA & Test efforts for the PIVX network. As we’ve seen in the past few months, the QA team have been actively working with the Development and Support teams to assist in the test efforts for 3.1.0.
There were items that were not achieved in the original proposal and items that were. Details of the previous proposals efforts can be found at the bottom of this proposal.
Testing scope of applications:
Tools:
For this next round of test efforts, the QA team will use TestRail. Test Rail is a test management tool for tracking tests executed and progress. Test Rail also has GitHub integration which is used for issue / bug tracking.
Infrastructure & Test Network:
From the previous proposal, we have set up 60 masternodes to support the PIVX testnet.
Sparrow had been putting a large amount of effort / days into ensuring the masternodes are running / up to date / on the correct chain or running new versions of the wallet. This effort had been underestimated in the previous proposal yet Sparrow had been doing the job without complaint. With this proposal, there will be provisions for his time and his costs of running the masternode network.
Test activities conducted, so far:
Sieres - Test Lead I will be leading and managing the test efforts for the PIVX QA team. I am the POC between QA and Dev and have been vetting, reproducing and reporting bugs that have been found by the QA team. To maintain security as a trusted PVIX team member, I provide binaries and updates to sparrow who then rolls out updates to the PIVX QA test machines.
In this model, I have been personally running functional tests from the PIVX private repository with the QA team validating results & regression testing.
Rahul – QA x 1
Rahul is a QA and TM that I know personally that I had brought on board to assist with the testing efforts. Rahul has been within the QA space throughout his entire career (10+ years) and has been assisting me in building and running the PIVX regression suite.
Strontium – Jnr QA x 1
Strontium is fast becoming a recognised member of our community, he has been taught by me in the technical side of PIVX and has fast learnt about PIVX through writing our whitepaper. He remains a Jnr Tester whilst I train him on test approaches, principles and practices.
TBC - Senior Test Automation Specialist x 1
I have identified an experienced automation specialist locally, hes role will be to automate and document a PIVX functional test suite. The identified resource is an ex colleague of mine and has been in the space of automation testing for over 10 years.
Sparrow - 1x Server & Environments manager
Sparrow is a known and trusted member of the PIVX community, he runs PIVXHost and has now expanded into running MyMasternode. Sparrow is passionate about helping the QA efforts in any way he can and has over extended himself to be as accommodating as he can possibly be. This has included daily and half daily builds / masternode updates / chain resyncs throughout the QA test efforts.
Deliverables:
Based on effort endured with the previous test proposal, I have adjusted salaries accordingly. Salaries are also accounting for the drop in value of PIVX since January.
I believe the previous proposal had been a success and on the back of that proposal I am submitting this proposal for a 6 month engagement for the PIVX QA team.
Initial Proposal Updates:
Title: pivx-qa-renewal
To Vote in favor or Yes:
mnbudgetvote many 6abf5a987615504914575d149b702ca1d4c4ced7b26e92765e22135fd2233fa3 yes
To Vote against or No:
mnbudgetvote many 6abf5a987615504914575d149b702ca1d4c4ced7b26e92765e22135fd2233fa3 no
To check the Vote status: getbudgetinfo pivx-test-team-renewal
2
May '18
Sieres Reporting in, with the next round of QA Proposals,
This proposal is to continue the QA & Test efforts for the PIVX network. As we’ve seen in the past few months, the QA team have been actively working with the Development and Support teams to assist in the test efforts for 3.1.0.
There were items that were not achieved in the original proposal and items that were. Details of the previous proposals efforts can be found at the bottom of this proposal.
Testing scope of applications:
- PIVX Core wallet (OSX / Windows) binaries, as provided by the PIVX core development team
o An effort will be made to setup the test environment to compile these, removing the dependency on the development team - PIVX Core wallet (linux, Ubuntu) compiled by the QA team
o All in-flight scenarios will be covered under this item (ie. Spending zPIV v1) - Rebaseline and increase the regression test suite to cover off post main net launch issues and new functionalitiy introduced with 3.1.0.2
- To automate the regression test suite (we found this was needed to be ran manually almost twice a week during the QA process)
- PIVX Core mobile wallet (where requested)
- PIVX Master node tool (where requested)
- PIVX Lite wallet (where requested)
- PIVX zDEX (where requested)
- Any future PIVX official applications that support the PIVX project
- Stretch goal: To build a pipeline that compiles the source code hooking into Test Rail and the automated test suite
Tools:
For this next round of test efforts, the QA team will use TestRail. Test Rail is a test management tool for tracking tests executed and progress. Test Rail also has GitHub integration which is used for issue / bug tracking.
Infrastructure & Test Network:
From the previous proposal, we have set up 60 masternodes to support the PIVX testnet.
Sparrow had been putting a large amount of effort / days into ensuring the masternodes are running / up to date / on the correct chain or running new versions of the wallet. This effort had been underestimated in the previous proposal yet Sparrow had been doing the job without complaint. With this proposal, there will be provisions for his time and his costs of running the masternode network.
Test activities conducted, so far:
- Test efforts on 3.1.0
- Test efforts on SPMT (PIVX Masternode Tool)
- Test efforts on current main net functionality (regression tests)
- POC on Test Tooling (TestRail)
- Documentation of the test strategy (still in draft)
- Creation of discord testnet bot automation (shells created)
Sieres - Test Lead I will be leading and managing the test efforts for the PIVX QA team. I am the POC between QA and Dev and have been vetting, reproducing and reporting bugs that have been found by the QA team. To maintain security as a trusted PVIX team member, I provide binaries and updates to sparrow who then rolls out updates to the PIVX QA test machines.
In this model, I have been personally running functional tests from the PIVX private repository with the QA team validating results & regression testing.
Rahul – QA x 1
Rahul is a QA and TM that I know personally that I had brought on board to assist with the testing efforts. Rahul has been within the QA space throughout his entire career (10+ years) and has been assisting me in building and running the PIVX regression suite.
Strontium – Jnr QA x 1
Strontium is fast becoming a recognised member of our community, he has been taught by me in the technical side of PIVX and has fast learnt about PIVX through writing our whitepaper. He remains a Jnr Tester whilst I train him on test approaches, principles and practices.
TBC - Senior Test Automation Specialist x 1
I have identified an experienced automation specialist locally, hes role will be to automate and document a PIVX functional test suite. The identified resource is an ex colleague of mine and has been in the space of automation testing for over 10 years.
Sparrow - 1x Server & Environments manager
Sparrow is a known and trusted member of the PIVX community, he runs PIVXHost and has now expanded into running MyMasternode. Sparrow is passionate about helping the QA efforts in any way he can and has over extended himself to be as accommodating as he can possibly be. This has included daily and half daily builds / masternode updates / chain resyncs throughout the QA test efforts.
Deliverables:
- Expansion of the Testnet Masternodes from 60 masternodes up to 150
- PIVX Test & Release Strategy: Stored on Github
- PIVX Test Suite moved to TestRail
- Test execution for all features in this proposal
- Test execution for core PIVX development items not listed in this proposal
- Test results stored in TestRail and available to key members of the community
o Due to licensing costs, access will be on an as needed basis - Test Automation Framework and Scripts: Stored on GitHub
Based on effort endured with the previous test proposal, I have adjusted salaries accordingly. Salaries are also accounting for the drop in value of PIVX since January.
- Sparrow: 250 PIV for Server maintenance / upkeep and expansion. This covers the cost of running the 150 strong masternode network and PIVX QA machines.
- 30 PIV for Test Rail, covers 6 users per month (https://bit.ly/2HzMMJ0)
- Sieres 900 PIV (25 hours+ / week) – Management, execution (including retest and defect management), planning, prep and development
- Rahul 450 PIV (10 – 20 hours / week) – Test tool setup and management, Test preparation and execution (including retests)
- Strontium 300 PIV (10 – 15 hours / week) – Test preparation, execution (including retests) & test documentation
- TBC - Automation developer / QA - 500 PIV – To build an IT industry standard functional test suite that is integrated into dev and test delivery. A resource has been identified and will be confirmed should this proposal pass
I believe the previous proposal had been a success and on the back of that proposal I am submitting this proposal for a 6 month engagement for the PIVX QA team.
Initial Proposal Updates:
- To introduce a formal round of feature testing to be conducted within the PIVX testnet.
o Update: This has been achieved - To ensure there is an adequate QA network infrastructure to support development of PIVX
o Update: This has been achieved - To create, implement and maintain a test strategy to be adhered to when moving through the development life cycle
o Update: This items remains in-progress. The document is rather detailed and is taking longer than originally anticipated. - To build a suite of test cases to cover new features that come through the development pipeline
o Update: This has been achieved with test cases covering zPIV / Budgets and Masternodes / zPIV Staking and other new features in 3.1.0 - To build a suite of regression test cases to cover off existing functionality that has not changed since the last minor or major build
o Update: This has been achieved. We have been running and re-running these tests of ~50 test cases once / week - To have a team of part-time QA’s employed to use testing techniques to test all test cases within the test suites AND conduct exploratory test sessions in an attempt to find edge case scenario bugs
o Update: This has been achieved - To report back and track any and all issues / bugs encountered with enough information to allow developers to triage and fix prior to a main net rollout
o Update: This has been achieved - To test in-flight scenarios and protocol enforcement when required
o Update: This has been achieved (testing v1 and v2 zPIV in 3.1.0) - A stretch target for this proposal will be to build a test automation framework and scripts to cover the core wallet functional operations
o Update: This had not been achieved, extended test efforts meant that this effort had been put aside. - An initial stretch target #2: implement Performance Test Capabilities for the test net
o Update: This had not been achieved, extended test efforts meant that this effort had been put aside.
Title: pivx-qa-renewal
To Vote in favor or Yes:
mnbudgetvote many 6abf5a987615504914575d149b702ca1d4c4ced7b26e92765e22135fd2233fa3 yes
To Vote against or No:
mnbudgetvote many 6abf5a987615504914575d149b702ca1d4c4ced7b26e92765e22135fd2233fa3 no
To check the Vote status: getbudgetinfo pivx-test-team-renewal