Difference between revisions of "Xen Document Days/TODO"

From Xen
(Removed historical done items)
Line 202: Line 202:
{{Comment|[[User:Lars.kurth|Lars.kurth]] 12:02, 2 April 2014 (UTC)}} This looks fine
{{Comment|[[User:Lars.kurth|Lars.kurth]] 12:02, 2 April 2014 (UTC)}} This looks fine
Would be useful to also talk about qemu and other repos (e.g. seabios, ovmf) and the branches they use.
For qemu see http://lists.xen.org/archives/html/xen-users/2014-01/msg00049.html
{{Comment|[[User:Ijc|Ijc]] 15:59, 27 January 2014 (UTC):}} Updated it a bit, now links to more of the repos, but could do with some more work.
==== (IMPORTANT) XenAPI OpenStack Documentation ====
See [https://blueprints.launchpad.net/openstack-manuals/+spec/redocument-xen related blueprint]
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 08:13, 1 April 2014 (UTC)}} These seem to be handled by OpenStack. Removing from tracker
==== Bootloader & Console Configuration ====
Surprisingly little on these two topics, which should be added to [[:Category:Host_Configuration]]
==== HOWTO : How to Set Up Nested Virtualization ====
See [http://lists.xen.org/archives/html/xen-users/2012-10/msg00311.html this thread]
* {{Comment|[[User:Dunlapg|George Dunlap]] 31 March 2014 (UTC)}} Renamed "Xen Nested" to "Nested Virtualization in Xen", expanded intro, added categories.
==== Sidebar rendering in Chrome (broken again) ====
See [http://stackoverflow.com/questions/21884833/old-mediawiki-site-broken-in-chrome]and [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=652948 (related)] and [https://code.google.com/p/chromium/issues/detail?id=28267 (chrome bug description) - note that this has been fixed, then re-appeared]
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 11:05, 31 March 2014 (UTC)}} Fixed by replacing KHTMLFixes.css on the server with an empty file. Before the change takes effect, clear the browser cache as per [https://support.google.com/chrome/answer/95582?hl=en-GB]
==== Xen 4.3 Release Notes ====
Need to flesh out [[Xen_4.3_Release_Notes]] (also see [[Xen_4.2_Release_Notes]])
==== Xen 4.3 new features that need documentation ====
Need to make a HOWTO for openvswitch, link it to 4.3 test day and 4.3 release notes wiki pages.
* {{Comment|--[[User:Lars.kurth|Lars.kurth]] 16:04, 24 June 2013 (UTC)} These existed before. Linked to from [[Xen 4.3 Feature List]
==== Xen 4.3 Man Pages ====
Need to flesh out [[Xen_4.3_Man_Pages]] (also see [[Xen_4.2_Man_Pages]])
==== Xen PV Architecture Documentation ====
Something like a SDM for Xen PV "processors".
Add pages to [[:Category:Paravirtualised Architecture]]
* [[X86 Paravirtualised Memory Management]]
** Needs OS specifics adding (e.g. Linux paravirt_ops)
* Grant Tables
* Event Channels
* I/O (blk in OK shape, net could catch up)
* ...
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 14:03, 29 January 2013 (UTC):}} [[Event Channel Internals]] was added
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 14:03, 29 January 2013 (UTC):}} Not sure how complete
* {{Comment|--[[User:Lars.kurth|Lars.kurth]] 10:58, 24 June 2013 (UTC)}} Enough of this has been done
==== A Xen Performance Tuning Guide ====
* Which workloads work for which virt mode. Network considerations. NUMA. VCPU Pinning. Etc.
* Examples from the KVM world: [http://sheepdog.taobao.org/people/zituan/kvm_tuning.html Guest perf tuning], [http://www.linux-kvm.org/page/Tuning_KVM KVM options affecting performance], [http://publib.boulder.ibm.com/infocenter/lnxinfo/v3r0m0/index.jsp?topic=%2Fliaat%2Fliaattunkickoff.htm Comprehensive guide covering pinning, caching, memory, huge pages, etc.]
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 19:50, 25 June 2012 (UTC):}} Work has been done on [[Tuning]]. Not sure whether more needs to be done.
* {{Comment|[[User:StefanoStabellini|Stefano]] 30 June 2012:}} The guide is in a decent state but I will keep improving it during the next doc days.
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 15:54, 19 September 2012 (UTC):}} Needs a quick sanity check for Xen 4.2
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 12:48, 22 January 2013 (UTC):}} Is this done?
* {{Comment|--[[User:Lars.kurth|Lars.kurth]] 09:55, 24 June 2013 (UTC)}} Looks like it. But rename document
* {{Comment|[[User:Rcpavlicek|Rcpavlicek]] 16:26, 15 May 2014 (UTC):}} Are modifications or additions required here? Given the work done over the past couple of years, I expected this to be a longer document.
* {{Comment|[[User:Ijc|Ijc]] 09:22, 21 May 2014 (UTC):}} The stuff about VCPU pinning seems to be a bit misleading, it's not always the best thing to do. See e.g. http://lists.xen.org/archives/html/xen-users/2014-05/msg00163.html and in particular http://lists.xen.org/archives/html/xen-users/2014-05/msg00170.html.
** In the referenced thread I said: "It does say "might" and "can" perhaps even those are a bit strong. Pinning is one tool in the performance tuning arsenal but it is very workload dependent on whether it will help or hurt (and it can be a lot in either direction).", which I think is accurate (and in any case the docs need to guide people better about when/whether to pin and what the up/downsides are etc).
==== Update Xen Release Features for Xen 4.3 ====
Add Xen 4.3 to [[Xen_Release_Features]] and create [[Xen 4.3 Feature List]] (also see [[Xen 4.2 Feature List]])
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 16:36, 29 April 2013 (UTC):}} [[Xen_Release_Features]] has been done based on [[Xen_Roadmap/4.3]]
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 16:41, 29 April 2013 (UTC):}} Created boilerplate for [[Xen 4.3 Feature List]]
==== Xen 4.3 Test Day Instructions ====
Need to flesh out [[Xen_4.3_RC1_test_instructions]] (also see [[XCP_1.6_test_instructions]] and [[Xen 4.2 RC2 test instructions]])
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 16:36, 29 April 2013 (UTC):}} Has been done. ARM support needs adding. Also george had some ideas.
==== A Xen Security Guide ====
The different options in Xen for Security, trade-offs, how to set them up, how to test Xen for security and how to optimize for different scenarios
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 09:48, 27 March 2012 (UTC):}} [[:Category:Security]] contains some security related docs
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 11:01, 4 April 2012 (UTC):}} It would be good if we could document a) XSM, b) Introspection API, c) Xen and SELinux in Dom0, d) Memory Access API (introduced in 4.1)
* User Docs that where migrating content would make sense and update such as (we can attach PDF's to the wiki now for starters, or we can use a [http://www.appropedia.org/Appropedia:Porting_PDF_files_to_MediaWiki conversion tool]
** http://bits.xensource.com/Xen/docs/user.pdf
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 15:59, 19 September 2012 (UTC):}} Some good articles came out of XenSummit NA (these can be built upon).
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 14:01, 29 January 2013 (UTC):}} Changes to [[Securing Xen]] & [[Driver Domain]]
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 14:01, 29 January 2013 (UTC):}} Got funding to document XSM; contract in place, but contractor is currently ill. This is the main outstanding issues.
* {{Comment|--[[User:Lars.kurth|Lars.kurth]] 07:57, 24 June 2013 (UTC)}} --[[User:Lars.kurth|Lars.kurth]] 07:57, 24 June 2013 (UTC)[[Xen_Security_Modules_:_XSM-FLASK]] has been added and I added an overview and removed the TODO
==== Categories are not returned in search ====
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 13:54, 29 January 2013 (UTC):}} Categories do not get returned when searching. To find a category, once has to go to Advanced, click Categories and search for say ''security''.
* We need to investigate whether there are better options for search (e.g. additional mediawiki plug-ins) or config options to change this behaviour
* '''IMPORTANT:''' This is actually a user preference. To change the default behavior go to [[Special:Preferences]] and tick ''Main'', ''Xen'', ''Help'' and ''Category''. I will investigate defaults.
* '''FIX:''' See [http://www.mediawiki.org/wiki/Manual:$wgNamespacesToBeSearchedDefault $wgNamespacesToBeSearchedDefault]
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 13:59, 3 May 2013 (UTC):}} Fixed
==== Help create an htaccess file from xen_org to xenproject_org ====
We need to create a mapping file from [http://xen.org/ xen_org] to [http://xenproject.org/ xenproject_org]. The plan is to archive [http://xen.org/ xen_org] to somewhere like http://xenproject.org/archive and to decide which pages in [http://xen.org/ xen_org] go to the archives, and which to the new site. I will track information at [[xen_org to xenproject_org mapping]].
==== Submitting Xen Patches ====
[[user:Dunlapg|George Dunlap]]: Updated [[Submitting Xen Patches]] to have recommended workflow for git; moved Mercurial-specifc instructions to [[Submitting Xen Patches - mercurial]] .
==== Document release signing keys and fingerprints etc ====
We should publish the key ids and fingerprints somewhere.
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 13:54, 29 January 2013 (UTC):}} See [http://www.xen.org/projects/xenhv_keys.html OpenPGP keys]
==== Better Statements on Compatibility ====
There seems to be some confusion in the wider Linxu community about XL/XM compatibility. Maybe some specific questions in an FAQ/the release notes. Also migration compatibility comes up often. Any articles related to compatibility should be listed in [[:Category:Compatibility]], which I am willing to clean up.
* {{Comment|[[User:Anthony|Anthony]] 14:33, 26 November 2012 (UTC):}} I add a bunch of pages to the category, and add few link in the category to upgrade from xend to xl.
* {{Comment|[[User:Anthony|Anthony]] 17:19, 26 November 2012 (UTC):}} I also redirect [[XL in Xen 4.2]] to [[XL]] as they had both the same content minus few things.
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 12:48, 22 January 2013 (UTC):}} Is this done?
* {{Comment|[[User:Ijc|Ijc]] 16:39, 25 January 2013 (UTC):}} I think we also need to cover compatibility confusion about 3.x vs 4.x.
* {{Comment|[[User:Ijc|Ijc]] 15:03, 28 January 2013 (UTC):}} I've added [[Xen Version Compatibility]] and a link from [[Xen FAW Compatibility]]. I think this is probably done for now?
==== Security related docs ====
[[user:Dunlapg|George Dunlap]]: Revamped [[Driver_Domain]] and [[Securing_Xen]]
==== cpupools intro / howto ====
[[user:Dunlapg|George Dunlap]]: See [[Cpupools_Howto]]
==== Clear up Xen TODO List ====
* [[Setting_boot_order_for_domUs]]
* [[XenDevelopmentProjects]] - although this will need maintaining
* Revamp http://wiki.xen.org/wiki/Xen_PCI_Passthrough. Should be more organized, easier to follow, less verbose, more thorough, and more up-to-date.
* Review and update http://en.wikipedia.org/wiki/Xen - in particular the "Host: Unix-like systems" is out-of-date. The Guest section is too.
* Make [[:Category:Host_Install|Host Install]] and [[:Category:Guest_Install|Guest Install]] more useful
** {{Comment|[[User:Lars.kurth|Lars.kurth]]:}} Note: both pages share [[Template:Distro_Resources]]
** Docs for many distros are missing: identify what docs are missing or out of date and put them on the todo list
* Feature Status Doc : [[Xen_Release_Features]]
* [[Xen Overview]] which explains the top level architectural features and options, trade-offs and links to examples tutorials that show how you set these up
* List i.e. stuff what to test after packaging. This would avoid the current state of affairs where most distros are infact broken w/re to Xen. See [[Distros]]
* Installation Guides: How to install on various distros. Tag with [[:Category:Host Install]] and [[:Category:Guest_Install]].
* vcpu pinning (part of [[Tuning]])
==== Document Policy on Maintenance Releases ====
Which releases are supported, for how long. Owners. As well as a mechanism for individuals (or vendors) to take ownership of older releases (as is the case for Xen 3.4).
* {{Comment|[[User:Ijc|Ijc]] 16:07, 26 November 2012 (UTC):}} Attempt to document current process: http://lists.xen.org/archives/html/xen-devel/2012-11/msg01389.html
* {{Comment|[[User:Lars.kurth|Lars.kurth]] 12:48, 22 January 2013 (UTC):}} Is this done, see [[Xen_Maintenance_Releases]]?
* {{Comment|[[User:Ijc|Ijc]] 14:01, 22 January 2013 (UTC):}} Doc is complete and patches to reference it are in the various trees.
==== Document {deb,rpm,tar}-pkg targets in kernel ====
[[Mainline_Linux_Kernel_Configs]] talks about the obsolete and Debian specific <code>make-kpkg</code> tool. Should reference the kernels own <code>foo-pkg</code> targets instead.
* {{Comment|[[User:Ijc|Ijc]] 12:12, 28 January 2013 (UTC):}} Done. Made a bunch of other cleanups too.

Revision as of 13:14, 25 June 2014

Icon Ambox.png The Xen Project Wiki has been subject to a severe spam attack in the last several months. To solve this and keep the wiki usable for everyone, we had to lock down the wiki and create an editors group. Please fill out this form and we will add you to the group.

Go back to Xen Project Document Days

Requested Documentation

Icon Info.png Feel free to comment on items using the following convention:
* {{Comment|~~~~:}} your comment

Feel free to add new workitems using the following convention:

==== Workitem ====
~~~~: work-item description

(SUGGESTED FOR MAY 28, 2014) Performance-related Documentation

The documentation on Performance Tuning seems thin, given the importance of the subject:

Missing bits in Beginners Guide

Xen_Project_Beginners_Guide has a lot of todos in it. In particular missing diagrams

(IMPORTANT) Linux Distros

It seems that the list of Linux Distros which support Xen is a lot larger than listed on this wiki.

The following pages need updating and new categories need to be created (these pages are used as reference by new users so they are extremely important to keep up to date!):

Pictogram voting comment 15px.png Lars.kurth 09:38, 5 June 2014 (UTC) Has all of this been done? It seems so

In progress (some just need double checking whether finished)

Draft a list of Questions for a User Survey

Restructure XCP Related Categories

NOTE: XCP is about to be transitioned to a new home, so it is important that we make sure the XAPI content remains after XCP goes. Correctly tagging items for XAPI is essential. [April 30, 2014]

Latest revision Your text

  • New Category XAPI Project : this is basically a category for developers
  • New Category XAPI Users : this is basically for users of XAPI, XAPI packages in distro and other flavours of XAPI (e.g. XCP ISO when not specific to the ISO)
  • Move content into those locations

Easy tasks for Beginners

You may want to read Wiki Help if you do not have an account.

Migrate useful blog posts to wiki

Sift through blog.xenproject.org and copy useful posts to wiki. This will mainly be a reformatting exercise. Before you copy an article, perform a search on the wiki (headline or key word) to see whether the post exists already. If in doubt, ask on the IRC channel.

Fix articles that need attention

Review Categorization

Pages With Comments or Improvement Suggestions

Missing Pages

  • Special:WantedPages - You can find missing pages by going to this link. There are several courses of actoin that can be taken:
    • The page is needed : Add {{TODO|With a comment}} to the top of the page
    • The page is not needed : Remove the reference to the page (follow the link in the bracket besides the page)
    • Note that the list of Wanted pages contains lots of needed translations and categories, which you can ignore
    • If in doubt, ask for help on the IRC channel

Pictogram voting comment 15px.png Double-p 09:55, 25 February 2013 (UTC): There are a gazillion Template-Whatever including a TODO, really needed? Pictogram voting comment 15px.png Lars.kurth 17:40, 25 March 2013 (UTC): See #Special:WantedPages_broken

Xen : Open work items

UEFI Secure Boot

Write a document on how to sign the xen binary for UEFI Secure Boot. Potential reference here: Dealing With Secure Boot

Documentation on lib(xen)vchan

The header has some stuff, but a more complete guide would be useful.

The usage of the example code is also not documented. http://lists.xen.org/archives/html/xen-users/2014-01/msg00162.html has some (incomplete) info.

Manpages for xe command

Write manpages for xe and its 430 subcommands.

Describe the different ways, and necessary components, to create PVs using xl

Raised by mail by Matthias Blankenhaus (who is prepared to take the lead, but needs help): This document would describe the different ways, and necessary components that each way requires, to create PVs using xl. Right now there are at least the following cases:

  • loopbacked vhds
    • single partition format
    • disk format
  • qemu backed vhds
    • qcow / qcow2
  • LVS backed image

This is not a complete list. Also the mileage of each case differs depending on the kernel features enabled in dom0/domu and also depending on the Xen version. Furthermore, certain format require additional infrastructure like the tap device.

All in all, this a complex topic that needs clarification. Doing this based on the lowest common denominator, xl, would clarify many things. In addition, baseline performance data on all of the above would help a great deal. Matthias volunteered to run the majority of needed perf tests

Review and Update: xenpm usage recommendations

There is Xen power management and Xenpm command which may all be outdated. These should probably be reviewed and updated.

HOWTO : using xm trigger to poweroff or firing off NMIs

Howto on using xm trigger to poweroff or firing off NMIs.

HOWTO: Chain pypxeboot and pygrub

How to chain pypxeboot and pygrub

  • Pictogram voting comment 15px.png Lars.kurth 10:35, 27 March 2012 (UTC): Is pypxeboot popular at all?
  • Pictogram voting comment 15px.png Fheigl: I added that. It is somewhat common, OVM 2 and OVM 3 rely on it for PXE installs. I'm not sure if xenpvnetboot is the same, or works. The use case is "you want to use the same installation server for Xen VMs and phys. servers. You want to be able to reinstall them based on PXE reply.
  • Pictogram voting comment 15px.png Fheigl: There is documentation for pypxeboot, but there is no (afaik) documentation for how pygrub is used as a fallback loader. We wasted many days just to get some hack that can't even load a newer kernel. Which sucks, in the end you spend more time/money making PV PXE boot work than you save by using virtualized servers.
  • Pictogram voting comment 15px.png Fheigl: How could it be popular given the state of documentation. <= Key issue to the left of this sentence.
  • Pictogram voting comment 15px.png Lars.kurth 16:03, 19 September 2012 (UTC): wondering whether this video contains what is needed
  • Pictogram voting comment 15px.png Ijc 11:01, 25 June 2014 (UTC): Although I've not watched the whole thing I stepped through that video enough to see all the slides (I think) and I didn't see anything about this topic or anything like it.
  • Pictogram voting comment 15px.png Ijc 11:17, 25 June 2014 (UTC): I've downloaded pypxeboot 0.0.3 and as far as I can tell the fallback to pygrub is supposed to happen automatically if pxelinyx.cfg says to do a local boot. If this doesn't Just Work then I think that is a code bug rather than a documentation issue. I'd recommend retiring this docs TODO in favour of a bug report to the pypxeboot authors and/or xen-devel. I googled around a bit and didn't find any existing bug reports anywhere.

Visually distinguish Xen, XAPI, MirageOS and Community Documentation better

Pictogram voting comment 15px.png Lars.kurth 15:32, 28 January 2013 (UTC): Users are getting confused when looking at wiki pages. If there was a visual way to distinguish Xen, XCP and Other content better, this would help. We could use colors and skinning when it comes to various elements, as well as masocts/logos:

  • For example: Help:Contents has a different feel (and colour scheme)
  • This could be replicated in trail boxes and other elements
  • We can also use mascots (note we don't have an XCP mascot)

But first, I need to investigate what is possible.

Pictogram voting comment 15px.png Lars.kurth 16:14, 28 January 2013 (UTC): Looks as if the only way this can be done is via Custom Namespaces. The only way to do this properly is to

  • Have different namespaces for Xen, XCP, ... and Community stuff and keep all the common stuff in Xen
  • The name spaces could then be styled differently, for example the Help namespace includes the ns-12 style into the body attributes, whereas the Main (default) namespace uses the ns-14 style.
  • Custom namespaces can be created by changing the config and are linked to custom styles
  • The styles are then defined in MediaWiki:Common.css

Implications of using custom namespaces:

  • Search: Pages in a custom namespace do not show up in default search results. So if we had a custom XCP namespace, it would mean that search results for XCP would not show up. One would need to go to Advanced Search and select XCP
  • Rename: Renaming pages is possible and as straightforward as renaming pages
  • Xen Namespace: The Xen namespace exists already, this may need to be renamed
  • Categories and Namespaces: Categories cross namespaces. In other words, Category:XXX would go across all namespaces. As an example, see Category:Wiki_Management

Mediwiki Issues

Special:WantedPages broken

Issue 1: Pages that need translation are spamming the list
  • Solution: fix language templates and mark with {{Stub translation}}
  • Pictogram voting comment 15px.png Lars.kurth 16:48, 28 January 2013 (UTC): There is something wrong with the language templates, essentially spamming the wanted pages. Do ignore these instructions until fixed.
  • Pictogram voting comment 15px.png Lars.kurth 23:28, 11 February 2013 (UTC): Fixed language template issues by removing most languages from the templates
  • Pictogram voting comment 15px.png Lars.kurth 18:03, 25 March 2013 (UTC): Started marking pages with {{Stub translation}}
Issue 2: Missing templates are spamming the list
Issue 3: Missing categories are spamming the list

Categories & Indexes

  • Pictogram voting comment 15px.png Lars.kurth 13:54, 29 January 2013 (UTC): Combined categories such as list all XCP FAQs have been asked for sevceral times
  • Investigate plug-ins

Developer Docs

Docs that need attention or are missing

  • Make XAPI docs (built from source) available on xen.org
  • PVOPS portal on xen.org - Konrad volunteered
  • some more xenstore docs (i.e. recent python bindings that were sent to the -users list)
    • Pictogram voting comment 15px.png Ijc 12:13, 25 June 2014 (UTC): I don't think these are included in Xen? Need a reference.
  • Using libxenstat python bindings (imo they're broken)
    • Pictogram voting comment 15px.png Ijc 12:13, 25 June 2014 (UTC): Is this a bug report? If they are broken we should fix them (as well as potentially documenting them)
  • More PV protocol docs (see recent patches to blkif.h for example)
    • Pictogram voting comment 15px.png Ijc 12:13, 25 June 2014 (UTC): blkif.h and netif.h are perhaps the most critical ones and those are now pretty good. But the rest (fbif.h, kbdif.h, pciif.h, usbif.h, tmpif.h and vscsiif.h) could all stand to be improved.
  • Document xenstore paths used by guests and toolstack etc

Done Recently

Xen 4.4 Documentation

Need to flesh out Xen_4.4_Acknowledgements

Xen 4.3 / 4.4 Documentation : Spice Config Example for upstream QEMU

Take this thread and convert into an Example

Pictogram voting comment 15px.png Lars.kurth 16:56, 9 April 2014 (UTC) Created SPICE_support_in_Xen for now

Update Xen Project Repositories

Xen Project Repositories references Xen 4.2 as the latest.

Pictogram voting comment 15px.png Rcpavlicek 23:59, 27 January 2014 (UTC): Updated reference to 4.3, but needs a developer to verify that nothing else is needed.

Pictogram voting comment 15px.png Lars.kurth 12:02, 2 April 2014 (UTC) This looks fine