Recently we had our online Ubuntu Developer Summit where we discussed a range of topics, defined next steps, and documented work items. The very last session at the event was an overall summary of the tracks (you can watch the video here), but I wanted to blog an overall summary too. These notes are quick and to the point, but they should give an overall idea of decisions made.
- Reviewed the current 13.10 release schedule found several changes made in 13.04 that mistakenly hadn’t been carried over, such as later freeze dates and one fewer alpha; Adam Conrad will be syncing all this up and sending mail to the ubuntu-release list for review.
- We discussed the positioning of the development release in light of some conversations last cycle, and put some more flesh on the design for making it easier for people to follow along with the development release all the time.
- This cycle, we’ll be bringing up a new 64-bit ARM architecture based on cross-building work done last cycle, and we’ll update developers on that once we get closer to the point of starting up builds in Launchpad.
- Moving forward with click packages. Fleshed out ideas on source package provision, integrating with existing client package management stacks, and clarifying some other things like the security model.
- For image based upgrades, the team held a demo and Q&A for the current proposed solution, which is split into client, server, and upgrader; client is going well and expected to land by the end of June, server is currently blocked on infrastructure but should be ready around the same time, and Ondrej Kubik has been making good progress at tweaking the CyanogenMod recovery environment for the upgrader.
- Firmed up the plan for packaging Android components for Ubuntu Touch images.
- Upstart will be used as the standard way of spawning desktop apps for Unity on touch devices and ideally on desktop too (Unity 7 and 8). This will let us make sure we only have one instance per app, and will make it easy to apply AppArmor, seccomp and cgroup confinement consistently to all apps.
- Defined a goal to reduce the amount of time it takes to prepare, test and make a Checkbox release, automating more of the process. This will benefit people who use the Checkbox tool as part of their daily work. It’s possible that Checkbox may move to Universe, although this needs some more discussion.
- The server certification tools are being reengineered to use the new plainbox engine as their core. This will preserve the existing UI, but we’ll have co-installable packages with the new core, and will eventually switch over to the new tools.
- The cert tools and test suite are being upgraded to work well on ARM for our hyperscale and mobile work, fixing any issues so we can get full, clean test runs on ARM servers. MaaS will be used for provisioning, and tested as a part of the ARM server solution.
- We will be basing the primary kernels for 13.10 on Linux 3.10, but will strongly consider 3.11 depending on timing. For Ubuntu Touch devices, we already have kernels available for Nexus 4 and 7, and plan to also bring up kernels for Galaxy Nexus and Nexus 10. We’ll provide a 13.04 hardware enablement kernel in the 12.04.3 point release.
- In terms of Ubuntu Touch power management, we have some preliminary baselines against Android on Nexus 4 and will replicate those on other devices, although they won’t be entirely meaningful until things like Mir land. We’ve written some new utilities such as eventstat to track down problems here. On power management policy, we agreed key requirements for the system power manager and we’ll extend powerd to serve our immediate needs.
- Approved LoCo teams are no more, will be verified teams.
- Bringing back fortnightly leadership meetings.
- Ubuntu Advocacy Kit is driving to 1.0.
- Gathered UDS feedback.
Ubuntu Community Website
- Great discussion which clarified everybody’s involvement in the project.
- Clear roadmap for completing the content and design in the next few weeks.
- Design and web team have the templates we need to finish the work.
- No discussion with IS yet around deployment – this will be coordinated next week.
Ubuntu Womens Session
- Several events planned to get more people involved and the word out (Career Days, UOW, etc.).
- Discussion about a women in technology themed event at CLS.
Ubuntu Status Tracker
- The status tracker is many things to many different teams, but we managed to figure out a number of issues we can tackle, which should make everybody’s lives easier.
- Removal of kanban view.
- Add links from team pages to milestones pages.
- Set up a meeting to discuss setting up an “ongoing” dev series.
Ubuntu On Air! Discussion
- Issues with supporting multiple hosts.
- Discussion about building support into summit and re-using vUDS components to support more shows and multiple hosts.
- We want to open it up to more contributors, so we get more variety into the shows.
Development Onramp for Touch / Unity Next
- Goals to improve docs.
- We will track contributions to all the projects to see how we improve.
- Increased focus on testing, coordination with the SDK team.
- Update Getting Started Page, review current docs and previous mailing list feedback.
- Regular doc review cadence and more health check meetings.
- Focus on content in the UAK.
Ubuntu Enterprise Desktop Discussions
- Another meeting will be planned to get more input from users of enterprise desktops.
- Some common issues were identified and discussed:
- outdated cfengine package
- access to Firefox/Thunderbird packages before publication (resolved)
- support for livemeeting/linc
More Ubuntu Touch images
- We identified the current blockers and will simplify thingsby using an image without firmware blobs, so they can be added by a local tool afterwards.
- After the rebase to saucy we will also update the docs accordingly.
- Kernel images for devices will first live in PPA, afterwards probably in universe.
Regular Ubuntu Development Updates
- Organise regular Ubuntu on Air Hangouts to which we invite people from news sites as moderators.
- Briefly summarise work from the last week(s).
- Ask engineers to demo/showcase interesting developments.
- Do Q&A sessions.
- Also invite members of governance teams along.
Cloud and Server
Openstack Next Steps
- Looked at some high level areas for this cycle, avoiding digging into areas covered by other sessions. We decided that at current, moving over to Git for our packaging work doesn’t add value. We also agreed to clean up on some cruft within the packaging branches.
Cloud Archive Status Check
- Decided we had to support Grizzly for 12 months, which exposes a 3 month support gap from the backing Raring release. Need to discuss with the security team about how to fill this gap. Reviewed proposal for SRU cadence and tentatively rubber stamped.
- Better co-ordination around trumping by Security dates, specifically if it covers more than one project.
- Looked at using updates as a reason to increase our messaging.
12.04.x images with LTS Enablement Kernel
- The cloud images currently only contain the Precise (3.2) kernel. Discussed adding the kernel HWE stack to cloud images. We need to document how to enable backports, clearly state the support, and possibly tool cloud-init to handle updating the kernel on boot if folks need a more recent kernel on boot.
- We will not be creating new images with the HWE kernel for the default images. The HWE kernel will be used for Clouds that have a high velocity of change in the Hypervisor (i.e. Windows Azure). For the regular images, we will investigate tooling in cloud-init and other places to make the ingestion of the HWE kernels easier, such as enhancing the documentation, allowing for easier enablement of backports, and making it easier to enable the HWE kernel at boot time.
Cloud-Init for Vagrant
- We will create a good Ubuntu development workflow for Vagrant users (cross platform OSX, Windows). Ben Howard will investigate cloud-init tooling as well as the best method to enable the DKMS modules.
Cloud Init & Cloud Image Development for Saucy
- We will define the development work to improve cloud-init and cloud images for the saucy cycle.
- Discussed work on pre-cloud init phase, vendor hooks, cloud init plugin, and rebuding tools.
Juju Core Development
- 1.10 version of juju available in backports for 13.04, and should be available in precise backports soon.
- Look for releases in juju/dev ppa updating weekly, juju PPA monthly, and have stable release go into backports (couple of times per cycle).
- HyperV support is currently untested.
- VMWare support in charms, but not primary supported charms.
- We need a matrix to demonstrate interoperability and support of each variation.
- Need to work out additional hardening support.
- Building on our great history, moving away from per commit hardware testing to a more fluid multi virtualised separated environment, allowing greater interoperability testing. Hardware Cert term showed interest in getting more involved. The scope of this will be ratified when the interop matrix is created.
Flag Bearer Charms
- Will improve flag bearer charm integration testing.
- Implement list of reference charms.
- Develop Percona backup XtraBackup flag bearer charm.
- Document flag bearer and reference charm criteria in best practices.
- Discuss flag bearer charms on mailing list.
Charm Policy Review
- Add into policy for a charm to provide a config option to specify the version. The other items such as installation location (ie /srv), implementation of common subordinates, backups are to be added to best practices. The 3 ack on charm reviews is still under discussion.
- Split Juju docs best practices and policy sections.
- Discussed re-reviewing the current charms in the charm store to ensure accurate readmes, tests, functionality, rating, categories, and icons. The workflow was discussed for queues, and which charms to tackle first.
Charm Development Tooling
- Discussed gathering all the different charm development tools into one central package. These charm development tools include charm-tools, charmsupport, juju-gui,openstack-charm-helpers. Folks also discussed how the tools could be improved, and used as a singular set.
Juju Framework Charm for Server Application Technologies
- Discussed further building out of the Django, Rails, Node.js, and possibly Java.
Improve Juju Documentation
- Make a better user and charm developer experience for juju.ubuntu.com/docs. Discussed getting a permanent beta site going, methods to get documentation contributions. Hopefully a revamped docs will be in production in the next couple of weeks, and if not we’ll have a beta site very shortly (days).
Juju Charm Testing
- Currently jenkins.qu.u.c has graph testing showing reliable results. Marco will be landing integration soon (days), with a more formal testing framework to follow (weeks).
- Some ideas discussed were to gate charm store commits on testing, showing testing status in the GUI, and pre-deployment testing. Test examples will be made available along with a charm testing school.
Add User Feedback loops and Social Networking to Charm Store Charm Pages
- Discussed making sure users have a method to give and receive feedback on a per charm basis. We currently have social networking (+1s, Likes, Tweets) in addition to downloads, quality rating, bug links, and testing status. Some ideas were to get clarification from design on showing social networking numbers, as well as a ‘leave feedback’ form.
Juju GUI Development
- Discussed development done, and upcoming work. Covered ideas such as design, bundles, diagnostics, user data, juju feature parity, maintenance and support.
Improving QA for seeded server packages
- Established three distinctive areas of testing, these are upstream test suites which typically run at build-time, integration tests via dep8 and service level testing which often requires multiple nodes and is conducted using juju.
- We established that there is the regression test suite that can be included in many of the packages directly, with the requirement that we package some of the common ubuntu testing libraries.
- Discussed some areas of standardisation for dep8 testing.
Fastpath installer work for 13.10
- Established what FPI is, and the processes which are part of it.
- Fast Path installer will be delivered as a installable package in Ubuntu, most likely in python. The interface to it will we yaml formatted configuration.
OpenStack Charm work for Saucy/Havana
- Migrate all charms to be python based.
- Consolidation into new charm-helpers nextgen library.
- Complete SSL/HTTPS support into charms.
- Integration of wiki and help documentation, upstream series aligned with upgrading notes.
- Design around how proprietary+1 plugins will be integrated into core charms for Cinder and Quantum.
Investigate alternatives to mysql
- Agreed that the best course of action was to maintain mysql for this cycle, and try and support other flavours of mysql getting into Ubuntu via Debian.
Ceph activities for Saucy
- Dumpling release will be out in August (co-incides with FF for Saucy) so will be target for this release.
- Lots of incremental improvements in efficiency and performance, full RESTful API for RADOS Gateway admin features, block device encryption for data at rest.
- ceph-deploy (upstream cross platform deploy tooling) will be packaged.
- Implementation of more automated testing during Saucy.
HA Openstack charms V2
- Reviewed the current state of HA support in Openstack charms. Percona has volunteered to charm their offering, allowing great coverage by their mysql HA variant for active/active clustering.
- Work also on active/active and brokerless messaging options (ZeroMQ) and incremental improvements for service check monitoring in load balancers.
- Cluster stack (Corosync/Pacemaker) to be reviewed and upgraded for Saucy in preparation for 14.04.
MongoDB activities for Saucy
- File Main inclusion report for Mongo to support Ceilometer and Juju use cases. Raise a Micro Release Exemption (MRE) to the techboard, as point releases are bug fix only.
- Upstream armhf enablement patches. Re-sync with Debian. OpenSSL license exception.
Virtualization Stack Work for Saucy
- If debian libcgroup maintainer finds time, we’d like to merge cgroup-lite into libcgroup. For per-user configuration, first make it default-off optional, conditional on userns sysctl being enabled.
- LXC work is going well on track to 14.04 (and lxc 1.0) roadmaps. For this cycle, we’d like to get user namespaces enabled in the saucy kernel with a new off-by-default sysctl controling unprivileged use, and complete the ability to create and start basic containers without privilege; add console, attach and snapshot to the API, complete the create API function, and convert all of the lxc-* programs to use the API; write a libvirt driver based upon the API, and a patch to enable testing it with openstack; write loopback and qcow2 block device drivers; Get the subuid (user namespace enablement) patches into the shadow package; discuss with the community the maintenance of stable trees; improve the API thread safety; and work our distro lxc tests into the upstream package (akin to how it is done in netcf).
- In edk2, we want to contribute to the implementation of the ability to save and restore nvvars from the ovmf bios from qemu. We’ll fix the apparmor bug preventing the block device mounting in libvirt-lxc, which is blocking use of that feature by openstack.
- We intend to merge libvirt at least at version 1.0.6, qemu at 1.5, and hopefully xen 4.3. We’ll follow up on citrix’ plans for xcp. The blueprint lists additional xen work planned. We’ll also look into default use of openvswitch bridges in libvirt.
- Autopilot testcases written for ubuntu core applications will be checked to ensure they pass before auto-landing updates in the ubuntu touch images.
- The quality community team will help core application developers develop a suite of manual testcases for each ubuntu core application. These will be run as part of the verification process for the 1.0 stable release of each application.
- Add testcases so all default desktop applications for each flavor are covered.
- Expand and improve server testcases to allow more participation by those who might lack domain specific knowledge and/or hardware.
- Make available documentation more accessible by linking to it from the tools we use for testing, like the qatracker.
- Continue holding testing ‘how-to’ and knowledge sharing sessions during UDW, UOW, as part of UGJ, and on ubuntu on air.
- Add testing achievements to the ubuntu accomplishments project.
- Ubuntu Touch images will be smoke tested using the pending/current model already in use for other images. This ensures no image is published for general consumption that doesn’t pass a set of tests ensuring basic functionality of the image.
- Current Ubuntu Touch autopilot tests for the core applications will be investigated for use as part of these smoke tests.
- The concept of smoke test is going to be expanded to cover a no regression build.
- Autopilot 1.3 is now released and will be available in raring and saucy. No quantal support is planned. Precise support is being examined, but requires further investigation and backporting work.
- Autopilot developers will now be available on #ubuntu-autopilot — no need to always ping thomi!
- Planned tests for stressing mir to ensure good behavior during stressed conditions for things like OOM, memory leaks and race conditions.
- Stress tests targeted to be run as often as possible, but might be limited due to time constraints of wanting to run the tests over a longer period of time.
- UTAH will be expanded to include automated upgrade testing capabilities. UTAH jobs will be created for bootstrapping base images, for performing upgrades, and running post-upgrade tests. The old auto-upgrade-testing tool can still be used by flavors if desired.
- Create high-level views of the state of quality in ubuntu by aggregating results of test runs. This will allow for ‘problem’ areas within ubuntu to be more easily identified and targetted for further testing or investigation by interested parties. You can follow this work on the QA dashboard here.
- autopilot-gtk will now be maintained by the upstream QA team. Bug fixes and outstanding issues will be solved in order to allow for the autopilot desktop tests to run
- Once running properly, the autopilot desktop tests will become a part of daily image testing
- Continue development on umockdev to add support for more exotic networking tests (eg, 3G) and research sound testing
As ever, you can track progress on work items on status.ubuntu.com and we hope to see you at the next UDS in three months.