Blog Marcina Bojko

Linux,Windows,serwer, i tak dalej ;)

Posts Tagged ‘work

Centos 6/7 virtual machines and Production Checkpoints in Hyper-V 2016.

leave a comment »

As we may know – Microsoft introduced new way of doing snapshots/checkpoints in Hyper-V 2016. However term „production” is misleading, implying Standard checkpoints are not production ready – which is simply not true.
The biggest difference is that Production checkpoints are mostly used with VSS-aware applications (like MS SQL/Exchange, MS Windows itself) allowing them to flush/sync/commit changes to filesystem.

As a major difference – production checkpoints don’t save memory or cpu state, starting always with machine powered off after restore.

You can choose which way you want to do your snapshots here:

Selection_999(411)

Windows-based virtual machines have supported this since previous versions of integration services (2012 R2, 8/8.1) and from the start in case of Windows 2016/10. What about Linux-based, Centos 6/7 machines?

When installed out of the box, without any additional packages, trying to do a production snapshot of Centos 7 (with all updates) we got something like this:

Selection_999(410)

Quick how-to.

  1. If youre using external LIS (Linux Integration Services) from Microsoft, as an external package – remove it. It’s a piece of crap,  breaking kernels from time to time, packed with ‚latest’ errors and workaround rejected by linux kernel maintainers. It’s really not worth risk to have it installed: 
    yum remove microsoft-hyper-v kmod-microsoft-hyper-v

    or

    yum remove $(yum list installed|grep microsoft)
  2. Check if your Hyper-V offers all Integration Services for this VM.Selection_999(412)
  3. Check  and install hyperv-daemons
     yum info hyperv-daemons

    Available Packages
    Name : hyperv-daemons
    Arch : x86_64
    Version : 0
    Release : 0.29.20160216git.el7
    Size : 4.5 k
    Repo : base/7/x86_64
    Summary : HyperV daemons suite
    URL : http://www.kernel.org
    Licence : GPLv2
    Description : Suite of daemons that are needed when Linux guest : is running on Windows Host with HyperV

    yum install hyperv-daemons -y
  4. Enable and start services
    systemctl enable hypervfcopyd
    systemctl enable hypervkvpd
    systemctl enable hypervvssd
    
    systemctl start hypervkvpd 
    systemctl start hypervvssd 
    systemctl start hypervfcopyd
  5. Check status
    [root@centos7 ~]# systemctl status hypervkvpd
    ● hypervkvpd.service - Hyper-V KVP daemon
     Loaded: loaded (/usr/lib/systemd/system/hypervkvpd.service; static; vendor preset: enabled)
     Active: active (running) since Wed 2017-07-26 02:37:30 CDT; 14s ago
     Main PID: 3478 (hypervkvpd)
     CGroup: /system.slice/hypervkvpd.service
     └─3478 /usr/sbin/hypervkvpd -n
    
    Jul 26 02:37:30 centos7 systemd[1]: Started Hyper-V KVP daemon.
    Jul 26 02:37:30 centos7 systemd[1]: Starting Hyper-V KVP daemon...
    Jul 26 02:37:30 centos7 KVP[3478]: KVP starting; pid is:3478
    Jul 26 02:37:30 centos7 KVP[3478]: KVP LIC Version: 3.1
    [root@centos7 ~]# systemctl status hypervvssd
    ● hypervvssd.service - Hyper-V VSS daemon
     Loaded: loaded (/usr/lib/systemd/system/hypervvssd.service; static; vendor preset: enabled)
     Active: active (running) since Wed 2017-07-26 02:37:30 CDT; 27s ago
     Main PID: 3485 (hypervvssd)
     CGroup: /system.slice/hypervvssd.service
     └─3485 /usr/sbin/hypervvssd -n
    
    Jul 26 02:37:30 centos7 systemd[1]: Started Hyper-V VSS daemon.
    Jul 26 02:37:30 centos7 systemd[1]: Starting Hyper-V VSS daemon...
    Jul 26 02:37:30 centos7 hypervvssd[3485]: Hyper-V VSS: VSS starting; pid is:3485
    Jul 26 02:37:30 centos7 hypervvssd[3485]: Hyper-V VSS: VSS: kernel module version: 129
    [root@centos7 ~]# systemctl status hypervfcopyd
    ● hypervfcopyd.service - Hyper-V FCOPY daemon
     Loaded: loaded (/usr/lib/systemd/system/hypervfcopyd.service; static; vendor preset: disabled)
     Active: active (running) since Wed 2017-07-26 02:37:30 CDT; 44s ago
     Main PID: 3492 (hypervfcopyd)
     CGroup: /system.slice/hypervfcopyd.service
     └─3492 /usr/sbin/hypervfcopyd -n
    
    Jul 26 02:37:30 centos7 systemd[1]: Started Hyper-V FCOPY daemon.
    Jul 26 02:37:30 centos7 systemd[1]: Starting Hyper-V FCOPY daemon...
    Jul 26 02:37:30 centos7 HV_FCOPY[3492]: starting; pid is:3492
    Jul 26 02:37:30 centos7 HV_FCOPY[3492]: kernel module version: 1

    As a result:
    Selection_999(413)
    and in /var/log/messages

    Jul 26 02:43:27 centos7 journal: Hyper-V VSS: VSS: op=FREEZE: succeeded
    
    Jul 26 02:39:25 centos7 systemd: Time has been changed
    
    Jul 26 02:39:25 centos7 journal: Hyper-V VSS: VSS: op=THAW: succeeded

 

Written by marcinbojko

Lipiec 26, 2017 at 18:05

Napisane w work

Tagged with , , , , ,

5 serious issues/deal breakers with System Center Virtual Machine Manager.

System Center Virtual Machine Manager was Microsoft’s answer to VMWare’s vSphere. It’s Microsoft, so what could have gone wrong? It’s Microsoft – so everything.
Below is a list of most annoying things, some of them are so serious it makes you wonder – maybe Powershell is the answer? Seriously? In 2017, Microsoft, you FORCE everybody to use text console again?
In a moment of doubt we used to call it overgrown cancer over Powershell commands.

Let’s start, sorted by weight of crime:

Deal breakers:

1) Terrible things you cannot do in SCVMM but you can in Hyper-V Manager, Failover Cluster or Powershell like:

  • rename you machine when its powered on (sic)
  • change its MAC from Dynamic to Static other way like manually copy it character by character.
  • change booting order (sic) of machines and templates
  • select all Integration Services offered
  • change location of smart paging file
  • change affinity with cluster (high/medium/low/do not autostart)

and so on.

2) Console. Console is so terrible, that its sorry state is just good meme source.

First – console from Hyper-V Manager/FailoverCluster

Selection_670.png

Then from SCVMM

Selection_671.png

  • you cannot attach console and then power on machine. You HAVE to – power on machine, wait few sec for console button to be available then race through time to start it BEFORE OS starts. You have better chances of winning some of Grand Prix then finish the trick above on first run.
  • only actions you have is Reconnect and Send CTRL+ALT+Delete. Never working ‚Clipboard’ added in SCVMM 2016 requires you to paste text HERE, then it’s pasted in VM console
  • when it start before machine starts – you have to kill an application. It’s no good to use it ever again, it won’t ‚click’ with machine you’ve started. Exit? Something terrible may happen.

Selection_672.png

3) Requirements

  • MS SQL Server Standard or Enterprise. https://technet.microsoft.com/en-us/system-center-docs/system-requirements/sql-server-version-compatibility
  • 4 GB Ram required, 16 GB recommended (don’t even bother going below)
  • A lot of not-really-so-working tricks to use it to manage hosts from other domains, especially without 2 way trusts settled.
  • Price. With whole gang of System Center tools, prepare to be robbed in a daylight. Doesn’t matter you have no intention to use other components – you have to pay for it. You cannot just pick and buy needed component – you have to buy-and-pay with bulk.

4) GUI

  • General slowness of GUI, regardless of hosts number, running tasks, library sizes.
  • Jobs window – generally unusable with more than one admin or more than one job running- lots of informational comments. Important actions (like: who deleted or altered machine) quickly goes off the screen, covered by messages like: refresh was completed.
  • Oh, did I mention ‚Refresh’ habit? Learn it. Learn it, and let your fingers memorize this config, as you will be using it a lot.

Refresh is required almost on everything. In options like: you DID change something via Powershell and HV-Manager – I can understand, refresh may be required. But you will have to hit REFRESH before, in time, and after ANY action you would like to perform. If not – expect the worst. Virtual machine seems to be non responding on your commands? Maybe its locked for backup, maybe it hanged, maybe it migrated to another host – you have to refresh, refresh and refresh to persuade SCVMM that you have most recent data.

Sometimes even refresh doesn’t work. Like in recovery or cluster node failure, you shouldn’t count on SCVMM to update its status before timer reaches day or two. Take your time! Sometime you will have to reboot SCVMM to persuade it to have the latest data. So, when your action fails – search no more, VM is probably locked, on other host or powered off. SCVMM takes its auto-refresh very slowly.

  • General over complexity in Logical Network and Switches. It’s like you have to create every VLAN again, even if you’ve done it on dozen of network devices, fill variables like subnets, gateways. You have to group it all together and again, attach to every Hyper-V switch on hosts you have.
  • Adding you own custom fields and filling them is, again, over complicated and requires you to do a lot of scripting and scheduling them in a Windows manner.
  • You cannot add, change or sort fields like Operating systems. What Microsoft got you are values like this:
    • Microsoft Windows Server 2012 R2
    • 64-bit edition of Microsoft Server

Selection_673.png

  • Hyper-V integration Services are always few releases behind. It started to change with Windows 2016 and idea to install them via Windows Update.
  • Inability to rename vm folder when machine changes its name. This way you will have to do a Live Migration to rename folder.
  • Complexity of generated script.One will think generating a new machine is easy: New-SCVirtualMachine with a lot of parameters. No. Script is long, heavy, complex and tries to do things in complete different matter.
  • Templates – only way to refresh a template is to create it again, or replace vhdx in library, and do some internal tricks.
  • Inability to do anything with machine when the job is running – all fields are grayed out and you have to wait for jobs to end or fail.

5) Agent

  • if you’re lucky, agents are deployed ALMOST instantly, but adding host to SCVMM requires it to restart
  • if you’re not lucky, then in case of SCVMM upgrade, you will have to manually redeploy and reinstall all agents. Quite common I’d say.
  • [IMPORTANT] The mess agent leaves on filesystem is just legendary. Lets say we would like to migrate our machine from folder d:\vm to e:\vm.After migration (when we choose right option) we will got:

– empty files in d:\vm\machinename

– machine in e:\vm\machinename

Let’s say we would like to migrate it back for some reason

We will get:

  • empty d:\vm\machinename
  • empty e:\vm\machinename
  • machine in e:\vm\machinename (1)

And migration is just done twice. Do you see the pattern? After few migrations we have complete chaos on filesystems with a lots of empty, semi-empty, almost empty and ‚soon-to-be-empty folders’. You’ll end up with removing them manually – again, if you’re lucky.

  • locked folder after failed job. Yes, when you migration failed, you will end with d:\vm\machinename which you’re not able to delete. Sometimes it can be deleted after some time, sometimes after SCVMM/host reboot, sometimes never.

Above list, not fully completed can be seen in SCVMM 2012 R2 and SCVMM 2016 versions. It’s clear that SCVMM is not very high on Microsoft ‚to do’ list as same errors and mistakes are transferred to newer version and hunts us until this day.

 

UPDATE (1)
Changed from Requirements (Enterprise) to (Standard , Enterprise)

Written by marcinbojko

Luty 4, 2017 at 20:08

Napisane w work

Tagged with , , , ,

Does Foreman speak SQL? It does ;)

So, the question is – how to deploy and maintain farm of Microsoft SQL Servers? With different domains, install sources, roles, features. Should we create unattended installers for every single instance?

No, we shouldn’t.

We should use Powershell DSC – https://github.com/PowerShell/xSQLServer
With Foreman/Puppet and win_manage, we have something like:

The simplest way:

instance1:
  dsc_instancename: MSSQLSERVER
  dsc_sourcepath: "\\our.server.com\ourshare"
  dsc_sourcecredential:
    user: anonymous
    password: anonymous
  dsc_setupcredential:
    user: DOMAIN\someuser
    password: somepassword

Or, more sophisticated:

instance1:
  dsc_instancename: MSSQLSERVER
  dsc_sourcepath: "\\our.server.com\ourshare"
  dsc_sourcecredential:
    user: anonymous
    password: anonymous
  dsc_setupcredential:
    user: DOMAIN\someuser
    password: somepassword
  dsc_features: SQLENGINE
  dsc_forcereboot: true
  dsc_agtsvcaccount:
    user: DOMAIN\scvaccount
    password: somepassword
  dsc_sqlsvcaccount:
    user: DOMAIN\sqlaccount
    password: somepassword
  dsc_sqlcollation: SQL_Latin1_General_CP1_CI_AS
  dsc_sqlsysadminaccounts:
    - DOMAIN\someadmin
    - DOMAIN\someotheradmin
  dsc_securitymode: SQL
  dsc_sapwd:
  user: sa
  password: paaaswordisverysecure
  dsc_sqluserdbdir: D:\MSSQL\Data
  dsc_sqluserdblogdir: E:\MSSQL\Data
  dsc_browsersvcstartuptype: Disabled

Whole install from local source takes aprox. 400 seconds (with other OS related settings) to finish.

Written by marcinbojko

Styczeń 22, 2017 at 21:11

Raw, unedited The Foreman/Puppet + Win_manage.

Written by marcinbojko

Październik 29, 2016 at 17:25

Napisane w Uncategorized, work

Tagged with , , , ,

Puppet & The Foreman & Powershell DSC – The Road So Far.

During last few weeks I was able to push and heavy test puppet-dsc code in a lots of environments and setups.

We had sysprepped Windows Server 2012 R2 images (different versions, builds and setups), a lots of Windows 10 Professional Workstations (Original, 1511, 1607 builds), few Windows 8.1 Pro – really great statistic sample.

As for now:

  • Windows Server 2012 and Windows 2012 R2 – fully supported
  • Windows 8.1/10 (original)/10 (1511) – fully supported
  • Windows Server 2016/Windows 10 (1607) – unsupported due to parsing bug in Powershell 5.1 – Work in progress
  • Windows 7/8 – not tested
  • Windows 2008 R2 – not tested

Implemented modules:

  • Chocolatey – with features and sources support (adding, removing, modyfing)
  • DSC_WindowsFeature
  • DSC_WindowsOptionalFeature
  • DSC_Service
  • DSC_Environment
  • DSC_Group
  • DSC_xFirewall
  • DSC_Reboot

More code is coming, but this fine set allows you to deploy and manage a lots of types of servers and workstations.

Written by marcinbojko

Październik 7, 2016 at 19:47

Chocolatey i własne źródło pakietów.

Bez wątpienia – Chocolatey (https://chocolatey.org/) jest doskonałym zarządcą pakietów dla systemu WIndows – zwłaszcza w instalacjach DesiredStateConfiguration. Trochę przytłacza mnogość sposobów instalacji pakietów (w zasadzie, każdy pakiet potrafi być oddzielnym kawałkiem kodu), jednak rekompensuje to z nawiązką ich niskim skomplikowaniem.

Owszem, sporo pakietów jest w oryginalnym repozytorium, jednak nie wszystkie pakiety źródłowe posiadają publiczne linki, tak aby można je było umieścić w samym pakiecie.

Rozwiązaniem jest np. tworzenie pakietów w repozytorium https://www.myget.org/, a same binarki trzymać np. 2 miejscach – firmowy intranet oraz linki publiczne.

Ponieważ mnie zawsze brakuje wspomnianych pakietów przedstawiam wam własne źródło, gdzie postaram się uzupełniać przydatne i brakujące pakiety.

Kilka zasad:

  1. Wszystkie pakiety binarne trzymane są w moim Dropboxie – katalog /public/choco.
  2. Wszystkie pakiety binarne podpisane są SHA256
  3. Wszystkie pakiety są weryfikowane przez Virus Total a ich suma SHA256 MUSI zgadzać się z sumą z linka VirusTotal. W opisie pakietu widnieje ich suma kontrolna.
  4. Repozytorium jest publiczne – jeżeli chcesz umieścić swój pakiet – odpowiadasz za niego 🙂

Bez zbędnej zwłoki:

Samo źródło: https://www.myget.org/feed/Packages/public-choco

Nuget v2: https://www.myget.org/F/public-choco/api/v2

RSS źródła z pakietami: https://www.myget.org/RSS/public-choco

 

Jak dodać źródło do swojego Chocolatey’a?

choco source add -n=NAZWA -s"https://www.myget.org/F/public-choco" --priority=10

Jak zainstalować/aktualizować pakiet?

choco install nazwapakietu -y

choco upgrade nazwapakietu -y

 

Jak usunąć pakiet?

choco uninstall nazwapakietu -y

 

Jakie pakiety są w repo?

Na dzień dzisiejszy:

WPS-Office-Personal – https://www.myget.org/feed/public-choco/package/nuget/wps-office-personal

Zabbix-Agent3 – https://www.myget.org/feed/public-choco/package/nuget/zabbix-agent3

 

Pozostałe pakiety przerabiam z sum kontrolnych MD5 na SHA256 co zajmuje mi nieco czasu.

 

 

Written by marcinbojko

Październik 1, 2016 at 12:57

Linux Mint 17.1 i Napiprojekt

W pracy bardzo doceniam zainstalowanego na laptopie Linuksa, zwłaszcza wtedy gdy potrzebuję stabilnego systemu do diagnozy, projektowania lub rozwiązania problemu. Aplikacje, skrypty, UI -to wszystko powoduje iż w codziennej pracy admina/architekta nie ma sobie równych – bardzo ciężko osiągnąć mi podobną funkcjonalność na stacji roboczej z zainstalowanym systemem z rodziny Windows.

W domu, do tej pory ostoi Windows’a – zawsze wystarczał mi wyżej wskazany laptop. Do czasu zakupy dobrej konfiguracji z 2 solidnymi 24 calowymi monitorami.

Ostatnią przyczyną korzystania z systemu Microsoftu były już tylko gry – jednak czy z tego powodu warto utrzymywać cały OS z innymi narzędziami? Dzięki naciskom Valve i nadchodzącym Steam Machines, już ponad 30% tytułów dostępnych na moim koncie Steam posiada swoje odpowiedniki  Szybki dual-boot z Linuksem zdecydowanie potwierdził tą tezę.

Przenosiny (wyjątkowo szybkie, skopiowanie /home z laptopa i dorzucenie wymaganych repozytoriów i pakietów) to temat na inny artykuł. Z poprzedniego OS brakowało mi tylko łatwego dostępu do zasobów Napiprojektu i błyskawicznego dopasowywania napisów do plików wideo.

W Linux Mint 17/17.1 skorzystać możemy z repozytorium zawierającego ostatnią wersję Qnapi (1.6-rc2-1) dla używanej przez nas architektury lub pobrać plik .deb bezpośrednio.

add-apt repository ppa:patryk-prezu/ppa

Jeżeli mamy już zainstalowane Qnapi warto dodać 2 dodatkowe akcje na managera Nemo, pozwalające pobierać napisy dla wszystkich zaznaczonych plików, we wskazanych językach.

W katalogu /usr/share/nemo/actions tworzymy dwa pliki o nazwach i zawartości:

nazwa pliku: 98_qnapi_en.nemo_action

[Nemo Action]
Active=true
Name=Pobierz napisy EN z QNapi
Comment=Pobierz napisy EN z QNapi
Name[en]=Download EN subtitles with QNapi
Comment[en]=Download EN subtitles with QNapi
Exec=qnapi -l EN %F
Icon-Name=qnapi
Selection=any
Extensions=avi;mkv;mpg;mp4;asf;divx;mpg;ogm;rmvb;wmv

 

nazwa pliku: 99_qnapi.nemo_action

[Nemo Action]
Active=true
Name=Pobierz napisy PL z QNapi
Comment=Pobierz napisy PL z QNapi
Name[en]=Download PL subtitles with QNapi
Comment[en]=Download PL subtitles with QNapi
Exec=qnapi -l PL %F
Icon-Name=qnapi
Selection=any
Extensions=avi;mkv;mpg;mp4;asf;divx;mpg;ogm;rmvb;wmv

Menu_187

Written by marcinbojko

Marzec 13, 2015 at 12:40

Napisane w Uncategorized

Tagged with , , , ,

%d blogerów lubi to: